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Introduction 


Afferent  inputs  concerning  resultant  gravitational  and  self-generated  bodily  accelerations 
are  critical  to  the  efferent  control  outputs  one  makes  to  skeletal  musculature.  Afferent,  efferent, 
and  reafferent  signals  are  continually  integrated  to  permit  coordinated  activity  and  build  an 
accurate  mental  model  of  one’s  spatial  orientation.  The  perception  and  control  of  body 
orientation  evolved  mainly  in  response  to  self-generated  accelerations  unlike  those  encountered 
during  passive  vehicle  travel  or  other  unusual  motions  frequently  experienced  in  present  times. 
Unusual  accelerations  and  associated  sensorimotor  (e.g.,  visual-vestibular)  discordances  are 
common  during  aviation  operations,  sea  travel,  cross-country  land  travel,  moving-based  and 
centrifuge-based  training,  space  flight,  and  extra-planetary  surface  exploration.  These  situations 
induce  spatial  disorientation,  degraded  dynamic  visual  acuity,  motion  sickness,  and  difficulty 
concentrating  (Guedry  &  Oman,  1990;  Graybiel  &  Knepton,  1976;  Lawson  &  Mead,  1998; 
Lawson,  Smith,  Kass,  Kennedy,  &  Muth,  2003;  Gibb,  Ercoline,  &  Scharff,  2011;  Lawson, 
Rupert,  Guedry,  Grissett,  &  Mead,  1997;  Cowings,  Toscano,  De  Roshia,  &  Tauson,  2001; 

Lathan  &  Clement,  1997).  Even  during  active  voluntary  movement,  sensory  disturbances  may 
arise  when  the  movement  occurs  in  a  non-terrestrial  force  environment  (e.g.,  a  space  station,  the 
Moon)  or  when  the  moving  person  is  experiencing  the  aftereffects  of  specific  insults  to  the 
vestibular  system  (e.g.,  due  to  head  injury  or  vestibular  disease).  This  paper  concerns  the 
application  of  mathematical  modeling  to  the  study  of  orientation  perception  and  sensory  illusions 
associated  with  movement.  We  describe  past  modeling  efforts  and  our  effort  to  refine  the  state  of 
orientation  modeling. 

A  Brief  History  of  Orientation  Modeling 

Static  orientation  models  have  been  developed  (e.g.,  Correia,  Hixson,  &  Niven,  1968) 
that  describe  whole  body  tilt  perception  (which  typically  determines  body  control  motor 
commands)  and  vestibular  gaze  reflexes  (which  determine  functional  dynamic  visual  acuity) 
during  constant  acceleration  stimuli,  such  as  acceleration  due  to  gravity  during  body  tilt  or 
resultant  (centripetal  plus  gravitational)  acceleration  during  constant  velocity  centrifugation  (off- 
center  rotation).  Relatively  simple  models  of  orientation  during  head  movement  also  have  been 
devised  and  empirically  evaluated  over  the  years  (e.g.,  Guedry  &  Benson,  1978;  Lawson, 

Guedry,  Rupert,  &  Anderson,  1994). 

As  validated  models  of  semicircular  canal  (SCC)  and  otolith  dynamics  became  available 
in  the  20th  century  (Goldberg  &  Fernandez,  1971;  Fernandez  &  Goldberg,  1976a,  1976b,  1976c), 
it  became  evident  that  end  organ  response  alone  could  not  adequately  account  for  the  time  course 
of  the  vestibulo-ocular  reflex  (VOR).  Moreover,  the  duration  of  the  illusory  after-rotation 
sensation  and  the  post-rotatory  eye  movements  following  constant  velocity  earth  vertical  rotation 
was  found  to  extend  well  beyond  that  of  the  actual  SCC  afferent  firing  duration,  a  phenomenon 
that  is  called  velocity  storage  (Raphan,  Matsuo,  &  Cohen,  1979).  Likewise,  the  gradual  time 
course  of  the  somatogravic  illusion  (i.e.,  the  well-known  pitch-up  illusion  that  occurs  in  response 
to  forward  linear  acceleration)  could  not  be  explained  by  otolith  afferent  dynamics  alone.  Early 
attempts  to  model  these  effects  included  the  velocity  storage  models  developed  by  Robinson 
(1977)  and  Raphan,  Matsuo,  and  Cohen  (1977,  1979)  and  the  complementary  filter  model 
developed  by  Mayne  (1974). 
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More  advanced  models,  based  on  concepts  from  estimation  theory  in  engineering,  have 
been  developed  and  applied  primarily  in  the  field  of  vestibular  physiology  (Borah,  Young,  & 
Curry,  1978;  Merfeld,  Young,  Oman,  &  Shelhamer,  1993;  Merfeld  &  Zupan,  2002;  Haslwanter, 
Jaeger,  Mayr,  &  Fetter,  2000;  Vingerhoets,  Van  Ginsbergen,  &  Medendorp,  2007;  Selva,  2009). 
Borah,  Young  and  Curry  (1978,  1988)  developed  a  steady-state  Kalman  Filter  (KF)  to  model  the 
orientation  perception  of  a  human  riding  passively  in  a  vehicle.  Their  model  included  vestibular 
motion  cues  as  well  as  dynamic  angular  and  linear  visual  velocity  information.  While  the  Borah 
model  was  capable  of  predicting  responses  to  a  number  of  vestibular  and  visual-vestibular 
motion  paradigms,  the  linear  nature  of  the  KF  restricted  its  application  to  small  head  deviations 
from  the  postural  upright.  Pommellet  (1990)  modified  Borah’s  internal  model  and  implemented  a 
time  varying  Extended  Kalman  Filter  (EKF)  to  account  for  full  nonlinear  motion  dynamics.  The 
EKF  model  was  able  to  match  Borah’s  predictions  for  simple  stimuli;  however,  it  exhibited 
numerical  instabilities  during  more  complex  motion  profiles  involving  larger  estimated  tilt 
angles.  A  follow-up  EKF  study  by  Bilien  (1993)  investigating  the  vestibular  portions  of  the 
model  encountered  similar  difficulties  when  modeling  Coriolis  cross-coupling  responses. 

Kynor  (2002)  and  Selva  (2009)  developed  stable  implementations  of  the  EKF  (Selva  also 
developed  a  stable  Unscented  Kalman  Filter  (UKF)  version  of  the  Borah  model)  and  were  able  to 
successfully  model  a  number  of  nonlinear,  large  angle  perceptual  responses,  including 
centrifugation  and  the  pitch-up  illusion  (during  rapid  forward  acceleration  such  as  a  catapult 
launch  takeoff).  While  the  stabilized  model  was  able  to  reproduce  simple  visual  illusions,  such  as 
linear  vection,  its  integration  of  visual  sensory  pathways  did  not  appear  to  match  the  true 
architecture  or  behavior  of  the  Central  Nervous  System  (CNS).  For  example,  the  model  relied  on 
visual  velocity  information  (via  visual  flow  of  the  surrounding  visual  scene)  to  suppress  the 
somatogravic  illusion  in  the  light,  a  response  which  was  not  found  experimentally  (Tokumaru, 
Kaida,  Ashida,  Misumoto,  &  Tatsuno,  1998).  Results  were  also  highly  dependent  on  model 
parameter  assumptions  and  were  extremely  sensitive  to  even  small  deviations  in  the  assumed 
sensor  bandwidths  or  noise  covariance  matrices.  Finally,  the  model  failed  to  reproduce 
sensations  arising  from  contradictory  vestibular  sensory  information.  This  finding  led  Bilien 
(1993)  to  conclude  that  the  optimal  implementation  of  the  KF  and  EKF  may  actually  be  “too 
optimal”  to  model  the  central  nervous  system’s  spatial  orientation  processes. 

Merfeld,  Young,  Oman,  and  Shelhamer  (1993)  developed  a  useful  “Observer”  model  of 
human  spatial  orientation  based  on  the  state  Observer  framework  proposed  by  Luenberger 
(1971).  Merfeld’s  model  (Figure  1)  utilized  nonlinear  quaternion  mathematics  and  internal 
models  of  semicircular  canal  and  otolith  dynamics  to  solve  for  central  estimates  of  angular 
velocity,  linear  acceleration,  and  gravity.  By  empirical  adjustment  of  the  four  internal  weighting 
parameters  in  Figure  1  (k0J,  kfM,  kf,  ka),  this  model  was  capable  of  predicting  the  orientation 
illusions  that  occur  during  a  number  of  motion  stimuli,  including  constant  velocity  earth-vertical 
rotation,  off- vertical-axis  rotation  (OVAR),  and  post-rotational  tilt.  Refinements  by  Haslwanter, 
Jaeger,  Mayr,  &  Fetter  (2000),  Merfeld  &  Zupan  (2002)  and  Vingerhoets  et  al.  (2007)  provided 
further  model  validation  (see  Figure  2  for  a  brief  history  of  Observer  and  optimal  control  model 
development). 
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Figure  1.  Original  Merfeld  spatial  orientation  model  (Merfeld  et  al.,  1993). 
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Figure  2.  Timeline  of  development  of  velocity  storage,  observer  and  optimal  control  (KF,  EKF, 
UKF)  perception  models. 

Recent  Modeling  Efforts 

Newman  (2009)  attempted  to  add  visual  sensory  information  to  the  original  Observer 
model  framework  and  extend  model  predictions  to  include  orientation,  position  and  linear 
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velocity  estimates.  Newman’s  model  was  able  to  mimic  orientation  perceptions,  linear  and 
circular  vection,  rotation  in  the  light,  and  acceleration  in  the  light.  A  summary  of  the  motion 
paradigms  simulated  successfully  by  this  model  and  others  is  presented  in  Table  1. 

Table  1.  Perception  Model  Applications  &  Validation  Stimuli 
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Until  recently,  orientation  perception  modeling  has  not  been  applied  extensively  to  large- 
amplitude  or  dynamic,  multi-axis,  three-dimensional  motions  such  as  those  that  occur  during 
natural  movement  or  during  usual  aircraft  maneuvers.  The  Disorientation  Analysis  and 
Prediction  System  (DAPS)  developed  by  Creare  Inc.  implemented  the  stable  Kynor  (2002)  EKF 
algorithm  into  a  MatLab®-based  graphical  user  interface  (GUI).  DAPS  was  able  to  successfully 
predict  SD  during  two  A- 10  accident  scenarios.  Since  both  of  the  chosen  A- 10  accidents  were 
the  result  of  underestimation  of  roll  rate  or  roll  angle,  the  general  utility  of  DAPS  for  high-G, 
supra-threshold,  tactical-flight  analysis  remains  uncertain.  Additionally,  the  aforementioned 
limitations  of  the  Kynor  (2002)  EKF  model  should  be  considered  when  using  DAPS  to  estimate 
perceptions  during  high  otolith-canal  conflict,  especially  in  the  presence  of  visual  flow  sensory 
input. 


Small  et  al.  (2006)  of  Alion  Corp.  developed  a  Spatial  Disorientation  Analysis  Tool 
(SDAT)  that  included  a  sophisticated  user  interface,  rules  for  classification  of  classic  orientation 
illusions,  and  a  rudimentary  model  for  sensory  cue  interaction.  Attempts  to  incorporate  a  more 
accurate  Observer  model  of  human  perception  into  SDAT  proved  difficult  due  to  the  inherent 
differences  between  the  two  modeling  techniques  (Small  et  al.,  2011). 

Groen,  Smaili,  and  Hosman  (2007)  proposed  an  alternative  (non-Observer)  model  for 
human  orientation  perception  and  provided  a  toolbox  of  MatLab®  routines  and  Simulink®  block 
diagrams.  Although  comprehensive  and  well-documented,  the  result  was  ultimately  only 
accessible  to  expert  MatLab®  users. 

In  order  to  improve  upon  these  earlier  efforts,  a  new  perception  model,  the  Orientation 
Modeling  System  (OMS),  was  developed  and  integrated  into  a  GUI-based  program  called 
Perception  Toolbox.  The  OMS  is  an  extension  of  the  Merfeld  Observer  family  of  models  and  the 
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subsequent  visual-vestibular  topology  developed  by  Newman  (2009).  The  methodology  for 
extending  the  Merfeld  model  and  adding  the  visual  sensory  components  is  detailed  in  sections 
“The  current  modeling  effort”  and  “Visual-vestibular  interaction  model”  of  this  report.  The  OMS 
was  programmed  in  MatLab®  and  Simulink®  and  was  designed  specifically  for  direct  integration 
with  Perception  Toolbox.  Perception  Toolbox  is  a  suite  of  perception  research  and  visualization 
tools.  Perception  Toolbox  was  designed  to  improve  the  accessibility  and  functionality  of 
perception  model  use  during  sensorimotor  research,  and  applied  investigations  of  human  error- 
related  accidents.  Specifically,  Perception  Toolbox  attempts  to: 

•  Improve  visualization  techniques  of  dynamic  human  perception; 

•  Facilitate  comparison  between  existing  perception  models; 

•  Provide  a  broad  set  of  predefined  modeling  applications,  based  on  simulations  from  the 
laboratory  and  aviation  settings; 

•  Provide  specific  tools  to  better  visualize  the  complex  dynamics  of  the  Coriolis  cross¬ 
coupling  effect  (Guedry  &  Benson,  1978); 

•  Extend  model  predictions  to  better  account  for  and  visualize  the  G-excess  effect  (Guedry 
&  Rupert,  1991);  and 

•  Provide  general  tools  for  prediction  of  reflexive  vestibular  gaze  reactions. 

The  features  of  Perception  Toolbox  are  outlined  in  the  section  “Perception  toolbox,” 
beginning  on  page  15.  In  the  next  section,  we  discuss  the  key  features  of  the  model  itself  before 
returning  to  its  visualization  tools. 

The  Current  Modeling  Effort 


Vestibular  Model 

The  vestibular  core  of  the  OMS  is  based  on  Merfeld’s  Spatial  Orientation  Model  shown 
in  Figure  3  (Merfeld  et  al.,  1993).  Note  that  the  Merfeld  model  presented  in  Figure  3  has  been 
rearranged  so  that  it  resembles  the  presentation  format  of  Haslwanter  et  al.  (2000).  In  the  model 
in  Figure  3,  three-dimensional  vectors  of  linear  acceleration  (a)  and  angular  velocity  (<x>)  are 
input  to  the  model  in  a  head-fixed  coordinate  frame.  Angular  velocity  is  integrated  using  a 
quaternion  integrator  (f  q)  to  keep  track  of  the  orientation  of  gravity  (g)  with  respect  to  the 
head.  The  otolith  transfer  functions  (OTO)  are  modeled  as  unity  and  respond  to  the  gravito- 
inertial  force  (GIF)  (/  =  g  —  a). 

The  SCC  are  modeled  as  second-order  high-pass  filters  with  a  cupula-endolymph  long 
time  constant  of  5.7  s  and  a  neural  adaptation  time  constant  of  80  s.  Afferent  signals  from  the 
canals  and  otoliths  are  compared  in  the  CNS  “Observer”  against  expected  values  from  a  similar 
set  of  internal  sensory  dynamics  (SCC,  OTO).  The  resultant  error  signals  are  weighted  with  four 
free  parameters  (kM,  kf0J,  kf,  ka)  highlighted  in  green.  The  model  outputs  are  central  estimates  of 

linear  acceleration  (a),  gravity  (g),  and  angular  velocity  (<7>).  Note  that  hatted  variables  (  ) 
represent  estimated  states. 

The  original  Merfeld  topology  was  designed  to  process  only  vestibular  sensory 
information.  In  order  to  incorporate  visual  cues,  several  structural  modifications  and  extensions 
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were  required  (Figure  4).  These  modifications  are  described  in  the  section  “Orientation  angle 
calculation”  and  based  on  previous  research  conducted  by  the  author  (Newman,  2009). 


Internal  Model  of  Sensory  Dynamics 
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Figure  3.  Merfeld  et  al.  (1993)  Spatial  Orientation  Model. 


Internal  Model  of  Sensory  Dynamics 
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Figure  4.  Extended  vestibular  model.  Blocks  highlighted  in  red  and  green  have  been  added  to  the 
original  Merfeld  model  topology.  See  page  12  for  further  explanation. 
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Orientation  Angle  Calculation 


Although  the  original  Merfeld  model  provides  a  representation  of  gravity,  it  does  not 
provide  a  direct  calculation  of  the  roll,  pitch,  and  yaw  angle  of  the  simulated  subject.  We  can 
derive  true  roll  (0),  pitch  (0),  and  yaw  (0)  Euler  angles  from  the  quaternion  integrator  used  to 
keep  track  of  the  true  gravitational  vector,  ( g )  (see  Appendix  for  a  full  description  of  the 
quaternion  mathematics  used  throughout  this  report). 
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Likewise,  we  can  derive  estimated  roll  (0),  pitch  (0),  and  yaw  (0)  Euler  angles  from  the 
quaternion  integrator  used  to  keep  track  of  the  estimated  gravitational  vector  ( g ). 
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Hypothetical  Limbic  Coordinate  Frame 

Many  visual  and  vestibular  perceptions  require  erroneous  estimates  of  one’s  position, 
velocity,  and/or  acceleration.  Brownout  during  a  vertical  helicopter  landing  and  helicopter  rotor 
wash  are  examples  of  disorienting  situations  wherein  position  and  velocity  information  becomes 
confusing  or  erroneous.  In  order  to  estimate  perception  of  linear  velocity  and  position  with  the 
model,  we  must  integrate  our  estimate  of  linear  acceleration  in  the  proper  coordinate  frame  used 
for  human  navigation. 

To  accomplish  this,  we  assume  that  the  CNS  integrates  the  perceived  linear  acceleration 
vector  (a)  in  an  allocentric  reference  frame  oriented  to  the  local  vertical.  We  refer  to  this  frame, 
which  is  aligned  with  the  perceived  gravitational  horizontal,  as  the  “limbic  coordinate  frame,” 
since  a  variety  of  physiological  evidence  suggests  that  estimates  of  azimuth  and  direction 
originate  in  limbic  areas  of  the  brain,  including  the  hippocampus,  thalamus,  and  medial 
entorhinal  cortex.  Neural  coding  of  place  and  grid  cells  (Best,  White,  &  Minai,  2001;  Hafting, 
Fyhn,  Molden,  Moser,  &  Moser,  2005;  Calton  &  Taube,  2005;  Knierim,  McNaughton,  &  Poe, 
2000;  Knierim,  Poe,  &  McNaughton,  2003;  Oman,  2007),  along  with  orientation  and  wayfinding 
experiments  performed  in  1G  (Aoki,  Ohno,  &  Yamaguchi,  2003;  Aoki,  Ohno,  &  Yamaguchi, 
2005)  and  0G  (Vidal,  Lipshits,  McIntyre,  &  Berthoz,  2003;  Vidal,  Amorim,  &  Berthoz,  2004), 
suggest  that  natural  perception  is  designed  for  terrestrial  2D  navigation  with  reference  to  a 
gravitationally  upright  body  axis  (Vidal  et  al.,  2004). 
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This  limbic  coordinate  frame  is  defined  by  the  quaternion  vector  (q)  from  the  estimated 
gravitation  state  ( g ).  At  each  time  step  of  the  simulation,  the  estimated  linear  acceleration  vector 
is  transformed  to  the  limbic  coordinate  system  (represented  by  the  red  T-l  block  in  Figure  4)  and 
integrated  twice  to  obtain  estimates  of  position  and  velocity  (represented  by  the  red  transfer 
functions  shown  earlier  in  Figure  4).  For  a  detailed  description  of  the  quaternion  mathematics 
and  transformation  methods  employed,  the  reader  is  referred  to  the  Appendix. 

Path  Integration 

Integration  of  estimated  acceleration  (a)  to  estimated  velocity  (v)  is  accomplished  with 
leaky  integration  with  individual  time  constants  for  motion  about  each  limbic  coordinate  axis. 
The  transfer  function  for  this  leaky  integration  is  displayed  below. 
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A  standard  integrator  was  implemented  for  the  velocity-to-position  integration  to  ensure 
that  a  static  visual  position  input  would  result  in  a  dynamic  response  with  zero  steady-state  error. 
Leaky  dynamics  initiate  phase  and  magnitude  estimation  errors,  which  do  not  correspond  to 
perceived  reality  (e.g.,  displacement  estimates  for  sinusoidal  horizontal  translatory  motion 
should  remain  accurate  with  a  visual  position  reference).  The  transfer  function  for  this 
integration  is  displayed  below. 
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Horizontal  vs.  Vertical  Motion 

It  should  be  noted  that  time  constants  for  motion  within  the  horizontal  plane  differ 
substantially  from  those  associated  with  vertical  motion  along  the  actual  or  perceived  direction 
of  gravity  (x  =  16.66s  vs.  x  =  1.0s).  Position  and  velocity  estimation  within  the  horizontal  plane 
has  been  shown  to  be  fairly  accurate  for  a  range  of  motion  amplitudes  and  frequencies  (Israel, 
Grasso,  Georges-Francois,  Tsuzuki,  &  Berthoz,  1989;  Mittelstaedt  &  Mittelstaedt,  2001; 
Seidman,  2008;  Guedry  &  Harris,  1963;  Mittelstaedt  &  Glasauer,  1991;  Loomis  et  al.,  1993). 

Large-amplitude  studies  performed  in  helicopters  and  vertical  motion  devices  (Walsh, 
1964;  Malcolm  &  Jones,  1973;  Jones  &  Young,  1978)  have  shown  a  fundamental  limitation  in 
the  ability  of  humans  to  integrate  inertial  acceleration  cues  along  a  gravitationally  vertical  axis. 
Experimental  subjects  were  found  unable  to  correctly  indicate  the  magnitude  or  direction  of 
motion,  often  eliciting  phase  errors  of  180°.  While  aware  of  vertical  displacement,  subjects  could 
indicate  the  proper  direction  of  travel  only  slightly  better  than  chance.  To  model  these  large 
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phase  and  magnitude  estimation  errors,  the  leaky  integration  time  constant  about  the  perceived 
vertical  has  been  substantially  shortened  (Figure  5). 
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Figure  5.  Model  response  to  sinusoidal  vertical  (A)  and  horizontal  (B)  displacement  profiles. 
Horizontal  motion  is  perceived  fairly  accurately  with  a  slight  magnitude  misestimation.  Vertical 
motion  exhibits  larger  phase  and  magnitude  errors. 

Parameter  Adjustment 

An  additional  feedback  gain  (/q)  has  also  been  added  to  the  model  (highlighted  in  green 
in  Figure  4).  This  parameter  is  a  function  of  the  angular  velocity  feedback  gain  (k0J)  and  cannot 
be  set  or  modified  directly.  K1  is  required  to  make  the  loop  gain  of  the  angular  velocity  feedback 
loop  unity.  We  calculate  k1  as; 


The  loop  gain  for  the  angular  velocity  pathway  is  calculated  as  the  reciprocal  of  the 
above  equation,  kM/ (k0J  +  1).  Using  the  Merfeld  &  Zupan  (2002)  parameter  (kM  =  3),  the  loop 
gain  is  found  to  be  0.75.  This  gain  was  intentionally  set  to  mimic  the  70%  angular  VOR  response 
for  eye  movement  data,  yet  is  inconsistent  with  perceptual  responses  for  many  simple 
experiments  (e.g.,  static  tilt,  constant  velocity  yaw  rotation),  where  the  initial  response  to  sudden 
head  movements  is  veridical. 


Visual- Vestibular  Interaction  Model 


The  preceding  vestibular  model  extensions  described  in  the  last  section  (“Vestibular 
model  development”)  are  essential  prerequisites  for  the  addition  of  visual  sensory  information. 
We  describe  here  this  modified  visual-vestibular  sensory  interaction  model. 
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Visual  Observers 
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Internal  Model  of  Sensory  Dynamics 


Figure  6.  Block  diagram  representation  for  a  generic  visual  model  pathway.  See  text  for 
explanation. 

To  keep  the  model  structure  and  notation  consistent  with  the  original  Merfeld  model 
(Merfeld  et  al.,  1993),  each  visual  pathway  is  constructed  as  shown  in  Figure  6.  For  a  generic 
visual  pathway  V,  a  visual  input  (Vv)  is  processed  by  the  visual  sensor  (VIS)  to  generate  a  visual 
sensory  estimate  (aVv  ).  This  estimate  is  compared  (C)  to  an  expected  visual  sensory  estimate 
(i aVv  )  from  an  internal  model  of  the  visual  sensor  (VIS).  The  comparative  difference  (eVv  ) 
(sensory  conflict)  is  weighted  with  a  residual  weighting  parameter  (Kv  )  and  added  to  the  rate  of 

change  of  the  estimated  state  ( Vv ).  The  weighted  conflict  vector  is  added  to  the  derivative  of  the 
state  in  order  to  keep  the  visual  model  additions  consistent  with  the  structure  of  a  classic 
Luenberger  Observer.  Since  Merfeld  et  al.  did  not  include  an  integrator  in  the  forward  loop  of  the 
angular  velocity  feedback  pathway,  we  added  the  weighted  visual  angular  velocity  error  directly 
to  the  state  itself. 

Coordinate  Frames 

We  assume  that  the  visual  system  is  capable  of  extracting  four  visual  cues  from  its 
environment.  These  are  position  (xv),  linear  velocity  (xv),  angular  velocity  (a)v),  and 
gravity/orientation  (gv).  The  visual  input  variables  are  represented  by  three-dimensional  vectors 
in  a  right  handed,  orthogonal,  world-fixed,  frame  of  reference  (XW,  YW,  ZW).  To  ensure 
congruency  with  the  vestibular  model’s  head  and  limbic  coordinate  frames,  the  visual  cues  are 
transformed  to  their  respective  frames  of  interaction  prior  to  sensory  processing.  Visual  gravity 
and  angular  velocity  are  transformed  to  the  head- fixed  coordinate  axes  and  visual  position  and 
velocity  are  transformed  to  the  perceived  limbic  frame. 
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Visual  Sensors 


For  simplicity,  we  assume  that  the  visual  system  sensory  dynamics  can  be  approximated 
as  unity  for  both  static  and  dynamic  visual  inputs.  We  do  not  yet  distinguish  between  focal  and 
ambient  vision  or  account  for  visual  saturation  limits  for  linear  and  circular  vection  cues.  These 
are  planned  future  modifications  to  the  model.  The  current  simplified  visual  model  allows  for  a 
baseline  assessment  of  the  usefulness  and  practicality  of  Observer  theory  for  modeling  multi- 
sensory  interaction.  In  three-dimensional  space,  we  can  represent  each  visual  sensor  as  a  3x3 
identity  matrix.  Since  dynamic  inputs  illicit  a  sensation  of  motion  in  the  opposite  direction  of  the 
visual  field  (e.g.,  linear  vection  and  circular  vection),  the  dynamic  sensors  are  modeled  as 
negative  3x3  identity  matrices.  Each  sensor  transforms  visual  input  (xv  xv  cov  gv)  to  visual 
sensory  estimates  (aXy  d±v  dgv  aMv). 


Static  Visual  Cues 


Dynamic  Visual  Cues 
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Internal  Model  Sensory  Dynamics 


We  assume  that  the  CNS  possesses  accurate  internal  models  for  each  visual  sensor.  Since 
the  CNS  already  accounts  for  the  proper  direction  of  the  visual  estimate,  we  can  represent  all 
internal  models  of  visual  sensory  dynamics  as  positive  3x3  identity  matrices.  The  internal  model 

of  the  visual  sensors  transform  the  central  state  estimates  (xv  xv  gv  ojv)  to  expected  visual 
sensory  estimates  (aXy  dXy  dgy  d0Jy). 
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Error  Calculations 


A  sensory  conflict  vector  is  calculated  for  each  visual  input  based  on  the  relative  error 
between  the  actual  and  expected  visual  sensory  estimates.  The  visual  position,  velocity,  and 
angular  velocity  errors  are  calculated  through  vector  subtraction.  Each  error  is  represented  as  a 
vector  containing  an  individual  sensory  conflict  for  each  orthogonal  axis. 
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Xy 
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The  gravitational  error  requires  both  a  magnitude  and  directional  component.  We 
calculate  the  conflict  vector  between  the  actual  and  expected  gravitational  sensory  estimates  by 
computing  the  rotation  required  to  align  both  vectors.  For  the  directional  component,  we  use  a 
cross  product  to  calculate  a  unit  vector  perpendicular  to  the  plane  formed  by  the  two  vectors. 


e_ w  _  x 

\e9v\  I agy  x  adv\ 

For  the  magnitude,  we  use  a  dot  product  to  calculate  the  angle  required  to  align  both 
vectors  within  the  previously  calculated  plane.  Note  that  this  implementation  is  identical  to 
Merfeld’s  GIF  rotational  error. 


Complete  Orientation  Modeling  System  (OMS) 

The  complete  block  diagram  for  the  OMS  is  shown  in  Figure  7.  Static  and  dynamic  visual 
inputs  are  added  to  the  extended  vestibular  model  denoted  in  Figure  4.  Model  inputs  now  include 
static  visual  position  (xv),  gravity  (gv),  dynamic  visual  velocity  (xv),  and  angular  velocity  (cov). 
All  cues  are  centrally  combined  and  used  to  generate  internal  estimates  of  angular  velocity  (to), 
acceleration  (a),  velocity  (v),  position  (x),  and  gravity  (g).  The  four  new  visual  free  parameters 
are  highlighted  in  green.  Values  for  the  free  parameters  are  shown  in  the  section  “Parameter 
calibration.” 


This  space  is  intentionally  blank. 


Figure  7.  Orientation  Modeling  System  (OMS)  block  diagram.  Blocks  highlighted  in  red,  blue, 
and  green  have  been  added  to  the  extended  vestibular  model  developed  in  the  section  “Vestibular 
model  development.” 

Visual  Sensory  Switches 

All  four  visual  sensory  cues  can  be  enabled  or  disabled  dynamically  during  a  simulation 
via  a  series  of  sensory  switches.  Switches  are  useful  to  model  changes  in  visual  sensory 
information  that  develop  in  a  situation  being  simulated  (e.g.,  a  person  closes  or  opens  his  eyes,  a 
pilot  gazes  at  an  aircraft  attitude  indicator  or  suddenly  enters  brownout  visual  conditions). 
Disabling  a  visual  sensor  is  not  equal  to  simply  setting  visual  input  to  zero.  A  zero-value  visual 
velocity  input,  for  example,  can  be  interpreted  as  a  static  visual  scene  that  is  not  translating  with 
respect  to  the  subject.  This  is  quite  different  from  a  scenario  in  which  a  visual  velocity  cue  is  not 
available.  Visual  sensory  switches  are  set  as  Boolean  time  history  vectors  in  the  input  data  file.1 

Visual  Weighting  Parameters 

Visual  error  signals  are  individually  weighted  with  residual  weighting  parameters  that 
can  be  adjusted  by  the  modeler  to  fit  the  data.  The  visual  gravity  residual  weighting  parameter 
( Kgv )  determines  the  influence  of  the  visual  gravitational  error  (egv  )  on  the  rate  of  change  of  the 

internal  estimate  of  gravity  ( g ).  The  visual  angular  velocity  residual  weighting  parameter  (K^y) 


1  See  section  “Perception  Toolbox.” 
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determines  the  influence  of  the  visual  angular  velocity  error  (ew  )  on  the  internal  estimate  of 

angular  velocity  (<n).  The  visual  position  residual  weighting  parameter  (Kxv)  determines  the 
influence  of  the  visual  position  error  ( eg )  on  the  rate  of  change  of  the  internal  estimate  of 

position  (x).  The  visual  velocity  residual  weighting  parameter  (Kxv)  determines  the  influence  of 
the  visual  velocity  error  (eXv)  on  the  rate  of  change  of  the  internal  estimate  of  velocity  (v).  The 
values  and  methodologies  used  to  set  the  weighting  parameters  are  detailed  in  the  section 
“Parameter  calibration,”  below. 

Parameter  Calibration 

Table  2.  Vestibular  Parameters 


Ka 

Kr 

Kco 

-4 

-4 

8 

8 

Table  3.  Visual  Parameters 


Kxv 

K±V 

K,v 

Kqjv 

0.75 

0.75 

5 

10 

The  OMS  has  a  total  of  eight  free  parameters  that  can  be  adjusted  to  tune  the  dynamic 
response  of  the  perceptual  estimates  (see  Tables  2  and  3).  The  four  vestibular  parameters  (Table 
2)  match  those  originally  calculated  by  Vingerhoets  et  al.  (2007).  Vingerhoets  et  al.’s  (2007) 
parameters  reflect  the  only  Observer  model  vestibular  weighting  scheme  validated  entirely  on 
perceptual  data,  and  were  thus  chosen  over  the  eye  movement  based  parameter  sets  of  Merfeld  et 
al.  (1993),  Merfeld  and  Zupan,  (2002)  and  Haslwanter  et  al.  (2000).  Parameters  were  tuned  to 
minimize  the  sum  of  squares  error  (SSE)  between  model  output  and  recorded  translation 
perception  during  OVAR.  A  limited  parameter  space  search  method  was  used  to  test  all  possible 
vestibular  parameter  combinations  within  the  Set  A,  shown  below. 

Set  A  =  [—8  -4  -2  -1  -0.5  0.5  1  2  4  8] 

A  general  stability  analysis  was  conducted  to  determine  the  range  of  valid  visual 
parameter  values.  Visual  parameters  were  adjusted  in  increments  of  ±  0.1  during  a  series  of 
simple  visual  sensory  paradigms  (i.e.,  linear  vection,  circular  vection,  static  orientation  cue,  and 
a  basic  forward  translation  simulation).  As  visual  parameters  became  marginally  or  completely 
unstable  the  resulting  perceptual  estimates  would  contain  large  amplitude  overshoots  or  high 
frequency  sinusoidal  oscillations  (shown  in  Figure  8B). 
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Figure  8.  Model  stability  assessment  during  a  roll  vection  stimulus.  (A):  Stable  roll  vection 
response.  (B):  Unstable  roll  vection  response.  The  unstable  response  exhibits  oscillations  that 
increase  in  frequency  as  the  simulation  progresses. 

Calculated  stability  ranges  are  shown  in  Table  4.  The  visual  position  parameter  ( Kxv )  and 
the  visual  velocity  parameter  (Kiv)  were  tuned  to  match  the  linear  vection  data  of  Chu  (1976). 
(See  section  “Linear  vection.”)  The  visual  angular  velocity  parameter  (K^)  was  tuned  to  match 
experimental  data  for  circular  vection  and  rotation  in  the  light  (Waespe  &  Henn,  1977).  (For 
further  analysis,  see  the  section  on  “Earth  vertical  rotation  [dark,  light]”  and  “Earth  vertical 
rotation  [circular  vection].”)  The  tuning  process  was  accomplished  with  a  trial  and  error  method 
based  on  the  pertinent  data  characteristics  responsive  to  parameter  adjustment  (e.g.,  rise  times, 
steady  state  values,  amplitudes,  and  phase  angles).  The  remaining  parameter  ( Kgv ),  which 
weights  the  strength  of  the  visual  orientation  error,  is  dependent  on  the  situation  being  modeled. 
A  sloping  cloud  bank,  fully  furnished  tilted  room,  or  simple  illuminated,  tilted  line  will  each 
produce  different  strength  orientation  cues  and  resulting  perceptions  of  tilt  (Singer,  Purcell,  & 
Austin,  1970).  The  modeler  should  consider  the  strength  of  the  visual  cue  being  modeled  and 
refer  to  the  stability  Table  (see  Table  4)  to  properly  set  Kgv. 

Table  4.  Stability  Ranges  for  Visual  Weighting  Parameters 


o  <  Kxv  <  1 

0  <  Kiv  <  1 

0  <  Kqv  <  330 

0  <Kmv<  178 

Perception  Toolbox  Input 

Perception  Toolbox  is  a  MATLAB®-based  program  developed  to  aid  in  the  processing, 
simulation,  and  visualization  of  human  perception  in  response  to  three-dimensional,  complex, 
multisensory  motion  stimuli.  Vestibular  and  visual  data  is  entered  into  the  model  through 
comma-separated-value  (CSV)  text  files  or  Excel  spreadsheets  (Figure  9).  These  data  files 
contain  time  histories  of  vestibular  and  visual  sensory  information,  head  movement  dynamics, 
and  sensory  switches  that  indicate  when  certain  cues  are  enabled  during  the  simulation. 
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Time 

Variable  name 

Coordinate  frame 

Unit 

Description 

Time 

n/a 

s 

Time  vector  denotes  rate  of  input 

Vestibular  Linear  Acceleration 

Variable  name 

Coordinate  frame 

Unit 

Description 

Ax 

Head 

m/s 

X-axis,  forward,  linear  acceleration 

Ay 

Head 

m/s 

Y-axis,  lateral,  linear  acceleration 

Az 

Head 

m/s 

Z-axis,  vertical,  linear  acceleration 

Vestibular  Angular  Velocity 

Variable  name 

Coordinate  frame 

Unit 

Description 

wx 

Head 

°/s 

Roll  angular  velocity  about  X-axis 

wy 

Head 

°/s 

Pitch  angular  velocity  about  Y-axis 

wz 

Head 

°/s 

Yaw  angular  velocity  about  Z-axis 

Visual  Position 

Variable  name 

Coordinate  frame 

Unit 

Description 

XV 

World 

m 

Position  of  subject  in  Xw  axis 

yv 

World 

m 

Position  of  subject  in  Yw  axis 

zv 

World 

m 

Position  of  subject  in  Zw  axis 

Visual  Velocity 

Variable  name 

Coordinate  frame 

Unit 

Description 

xdotv 

World 

m/s 

Velocity  of  subject  in  Xw  axis 

ydotv 

World 

m/s 

Velocity  of  subject  in  Yw  axis 

zdotv 

World 

m/s 

Velocity  of  subject  in  Zw  axis 

Visual  Angular  Velocity* 

Variable  name 

Coordinate  frame 

Unit 

Description 

wxv 

Head 

°/s 

Visual  roll  angular  velocity  about  X-axis 

wyv 

Head 

°/s 

Visual  pitch  angular  velocity  about  Y-axis 

wzv 

Head 

°/s 

Visual  yaw  angular  velocity  about  Z-axis 

Visual  Orientation 

Variable  name 

Coordinate  frame 

Unit 

Description 

Gxv 

World 

n/a 

X  component  of  "down"  direction  in  Xw  axis 

Gyv 

World 

n/a 

Y  component  of  "down"  direction  in  Yw  axis 

Gzv 

World 

n/a 

Z  component  of  "down"  direction  in  Zw  axis 

G  Magnitude 

Variable  name 

Coordinate  frame 

Unit 

Description 

g 

n/a 

G 

Magnitude  of  gravity  vector 

Visual  Sensory  Switches 

Variable  name 

Coordinate  frame 

Unit 

Description 

Pos  ON 

n/a 

Boolean  (0  or  1) 

Visual  position  cue  ON  =  1 ,  OFF  =  0 

Head  Movement 

Variable  name 

Coordinate  frame 

Unit 

Description 

0xh 

Head 

o 

Head  roll  angle 

0yh 

Head 

o 

Head  pitch  angle 

Ozh 

Head 

o 

Head  yaw  angle 

(s  =  seconds;  m_=  meters;  °  =  degrees;  G  =  G- units) 


*  The  input  for  a  rotating  scene  is  the  negative  of  the  vestibular  angle  velocity  input  since  vection  is 
induced  in  the  direction  opposite  of  the  visual  scene  motion. 


Figure  9.  Example  Excel  input  file. 
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After  a  file  has  been  loaded  into  the  main  GUI,  it  is  processed  through  a  Simulink®-based 
mathematical  perception  model  that  estimates  what  an  individual  would  perceive  given  the 
current  sensory  information  and  motion. 

The  Perception  Toolbox  has  three  tools  for  data  visualization:  2D  time  history  plots,  3D 
animated  vector  plots,  and  virtual  reality  simulation.  The  virtual  reality  (VR)  simulation  in 
particular,  is  a  powerful  tool  for  sensorimotor  researchers,  human  factors  engineers,  and  accident 
investigators.  Users  can  visualize  the  actual  motion  of  a  simulated  VR  avatar  side-by-side  with 
the  predicted  orientation  derived  from  the  perception  models. 

Two  additional  modeling  tools  are  included  and  accessible  from  the  main  GUI:  the  VOR 
Toolbox,  used  to  model  eye  movement  dynamics,  and  the  G-excess  Toolbox,  used  to  predict 
orientation  perception  under  various  acceleratory  and  gravitational  states. 

Perception  Toolbox  Graphic  User  Interface 

Figure  10  shows  the  Perception  Toolbox  main  GUI  window.  From  this  window,  a  user 
can  set  and  modify  all  model  parameters,  input  motion  data  files,  select  individual  models  for 
simulation,  export  perceptual  estimates  to  CSV  or  Excel  and  access  the  five  other  Perception 
Toolbox  visualization  and  data  processing  tools. 


Figure  10.  Main  graphical  user  interface. 

The  simulation  time  for  each  model  is  also  calculated  and  displayed  on  the  main  GUI. 
This  value  provides  a  rough  estimate  of  model  efficiency.  Efficiency  is  particularly  important  for 
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potential  future  applications  that  involve  real-time  integration  of  a  perception  model  into  a  given 
process,  sensor,  or  display  (Lawson,  McGrath,  Newman,  and  Rupert,  2015). 

Perception  Models 

In  addition  to  the  OMS  outlined  in  this  report,  six  other  classic  perception  models  have 
been  preprogrammed  into  the  Perception  Toolbox  (see  Table  5).  These  additional  models 
facilitate  comparison  with  previous  research  and  modeling  results.  To  our  knowledge,  this  is  the 
only  orientation  model  with  this  capability. 

Table  5.  Spatial  Orientation  Models  Included  in  Perception  Toolbox 


Models  Included 

Reference(s) 

Notes 

Orientation  Modeling 
System  (OMS) 

Newman,  2009 

Two  implementation  (with 
and  without  visual  cues) 

Merfeld 

Merfeld  et  al.,  1993 

Vestibular  cues  only 

Haslwanter 

Haslwanter  et  al.,  2000 

Vestibular  cues  only 

Merfeld  &  Zupan 

Merfeld  &  Zupan,  2002 

Vestibular  cues  only 

Vingerhoets 

Vingerhoets  et  al.,  2007 

Vestibular  cues  only 

Extended  Kalman  Filter2 

Borah  et  al.,  1978; 
Pommellet,  1990;  Kynor, 
2002;  Selva,  2009 

Two  implementation  (with 
and  without  visual  cues) 

Unscented  Kalman  Filter3 

Selva,  2009 

Two  implementation  (with 
and  without  visual  cues) 

Each  model  is  implemented  in  its  own  Simulink®  block  diagram  and  can  be  selected 
independently  for  simulation.  The  Perception  Toolbox  correctly  formats  input  data  for  each 
perception  model  and  stores  outputted  estimates  of  spatial  orientation  perception  in  MATLAB® 
structures  for  future  comparison  and  visualization.  The  parameters  and  feedback  gains  for  each 


2  The  family  of  Kalman  Filter  &  Extended  Kalman  Filter  (EKF)  models  that  began  with  Borah,  Young,  and  Curry’s 
(1978)  Sensory  Mechanism  Model  and  progressed  to  the  latest  nonlinear  models  by  Kynor  (2002)  and  Selva  (2009) 
all  maintain  a  very  similar  mathematical  foundation  to  solve  for  estimates  of  spatial  orientation  perception.  The 
more  recent  implementations  have  added  more  advanced  nonlinear  estimation  techniques  and  therefore  have  been 
chosen  for  implementation  in  the  Perception  Toolbox. 

3  The  Unscented  Kalman  Filter  (UKF)  model  implemented  by  Selva  (2009)  addresses  several  of  the  approximation 
issues  intrinsic  to  Extended  Kalman  Filters.  Unlike  the  EKF,  the  UKF  does  not  explicitly  approximate  the  nonlinear 
process  and  observation  models.  This  approximation  can  yield  very  large  errors  in  the  estimated  statistics  of  the 
posterior  distribution  of  the  states.  The  UKF  uses  the  true  nonlinear  model  and  an  unscented  transformation  for 
mean  and  covariance  propagation.  Accuracy  of  mean  and  covariance  estimates  for  the  UKF  are  third  order 
(assuming  Gaussian  statistics),  compared  to  only  first  order  with  the  EKF. 
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model  are  fully  customizable  via  the  main  GUI.  Default  parameters  correspond  to  those 
published  in  the  original  manuscripts  listed  in  Table  4. 

Plotting  Tools 

The  Plotting  Tools  GUI  is  shown  in  Figure  11.  From  this  window,  users  can  plot  the 
actual  motion  stimuli  against  estimates  of  acceleration,  angular  velocity,  gravity,  gravitoinertial 
force,  linear  velocity,  displacement,  head  angle,  head  velocity,  and  head  acceleration  for  each  of 
the  perception  models  listed  in  Table  4.  Users  can  plot  the  results  for  a  single  model  or  compare 
estimates  for  all  models  by  selecting  the  desired  checkboxes  in  the  plot  legend  side  panel.  More 
advanced  MATLAB®  plotting  features,  such  as  zoom,  data  tips,  regression  analysis,  and  plot 
scroll  are  accessible  in  a  convenient  toolbar.  Plots  can  be  printed  and  saved  to  image  files  for 
offline  analysis. 


Figure  11.  Plotting  Tools  graphical  user  interface.  Results  for  multiple  perception  models  are 
shown.  Each  trace  represents  the  response  of  an  individual  perception  model.  Trace  colors 
correspond  to  the  perception  model  listed  in  model  legend  in  the  right  center  sidebar  panel. 
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Vector  Visualization  Tools 


The  Vector  Tools  GUI  is  shown  in  Figure  12.  Vector  Tools  displays  animated  2D  and  3D 
plots  for  any  of  the  variables  calculated  by  the  Perception  Toolbox.  Users  are  presented  with  a 
2D  time  history  of  the  chosen  variable,  a  polar  plot  of  estimated  and  actual  azimuth  angle,  and  a 
vector  plotted  in  3D  vector  space.  Users  can  modify  the  perspective  of  the  3D  vector  plot  during 
the  simulation  to  isolate  the  XY,  YZ,  and  XZ  vector  planes. 


Figure  12.  Vector  Visualization  Tools  graphical  user  interface. 

Virtual  Reality  (VR)  Visualization  Tools 

Virtual  reality  playback  is  accomplished  using  MATLAB®’s  3D  Animation  Toolbox  and 
the  VRML  (Virtual  Reality  Modeling  Language)  3D  World  Blaxxon  Freeware.  The  VR  Tools 
GUI  is  composed  of  three  independent  virtual  worlds.  World  one  displays  the  actual  motion  of 
the  simulated  subject.  World  two  displays  the  estimated  motion  driven  by  a  selected  perception 
model.  World  3  provides  a  close-up  view  that  displays  head  movements  made  throughout  the 
simulation  (Figure  13D).  These  worlds  are  integrated  into  a  MATLAB®  GUI  as  shown  in  Figure 
13  A.  From  this  GUI,  users  can  select  the  perception  model  that  they  wish  to  visualize  and  set  a 
number  of  important  options  and  settings  for  VR  playback. 
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Figure  13.  Virtual  Reality  Visualization  Tools  graphical  user  interface.  (A):  Main  interface.  (B): 
Example  of  the  path  trajectory  visualization.  (C):  Example  of  the  Coriolis  tumbling 
visualization.  (D):  Example  of  the  head  movement  model  visualization. 

The  path  trajectory  visualization  option  (Figure  13B)  displays  the  motion  path  for  the 
simulated  subject  in  virtual  space.  The  Coriolis  tumbling  visualization  option  (Figure  13C) 
allows  users  to  view  the  paradoxical  sensation  of  limited  tilt  displacement  accompanied  by  a 
persistent  velocity  tumbling  sensation  that  is  often  reported  during  the  Coriolis  cross-coupled 
illusion.  A  similar  visualization  can  be  done  for  some  other  illusions  with  paradoxical 
displacement/velocity  characteristics,  such  as  a  roll  tilt  sensation  due  to  circular  roll  vection. 
Selecting  this  option  converts  the  estimated  orientation  into  two  individual  sensations:  a)  static 
tilt,  represented  by  the  solid  avatar  in  Figure  13C,  and  b)  tumbling  velocity,  represented  by  the 
translucent  rotating  avatar.  When  viewed  in  real  time,  this  visualization  tool  provides  a  powerful 
illustration  of  the  combined  perceptual  response. 
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Vestibulo-Ocular  Reflex  (VOR)  Tools 


Eye  movements  provide  another  physiological  measure  that  can  be  used  to  validate  and 
tune  a  perception  model’s  dynamic  response.  Unlike  verbal  reports  or  estimates  of  orientation 
that  rely  on  the  psychophysical  setting  of  a  vertical  line,  eye  movements  are  a  direct,  reflexive 
response  to  angular  and  horizontal  motion.  It  is  important,  however,  to  note  that  the  link  between 
orientation  perception  and  eye  movements  is  complicated  and  not  fully  understood.  What  a 
subject  perceives  or  reports  may  not  agree  with  eye  movement  response. 


Figure  14.  Vestibulo-Ocular  Reflex  Tools  graphical  user  interface. 

We  dissociate  eye  movement  predictions  from  spatial  orientation  perception  predictions 
using  a  separate  VOR  toolbox  (Figure  14).  The  VOR  Toolbox  uses  a  basic  eye  movement  model 
(Figure  15)  to  calculate  the  slow  phase  eye  velocity  of  the  VOR  (Merfeld  et  al.,  1993;  Merfeld  & 
Zupan,  2002).  VOR  is  modeled  as  the  sum  of  angular  VOR  (AVOR)  driven  by  the  perception 
model’s  estimate  of  angular  velocity,  and  translational  VOR  (TVOR)  driven  by  estimated  linear 
acceleration.  The  VOR  Toolbox  allows  users  to  plot  the  AVOR,  TVOR,  and  total  VOR,  and 
adjust  the  two  free  parameters  of  the  eye  movement  model. 
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Figure  15.  Merfeld  &  Zupan  (2002)  Vestibulo-Ocular  Reflex  Model. 

The  VOR  is  modeled  as  the  sum  of  the  AVOR  and  the  TVOR.  The  TVOR  is  derived 
from  the  estimate  of  linear  acceleration  by  leaky  integration  followed  by  a  cross  product  with  an 
estimated  target  proximity  vector  p,  which  accounts  for  distance  and  gaze  direction  influences. 
The  leaky  integration  time  constant  tv  is  set  at  a  default  value  of  0.1s.  The  proximity  vector  is 
equal  to  [1/d,  0,0],  where  d  is  equal  to  the  distance  from  the  given  focal  target.  By  default  d  is 
set  equal  to  2  meters  to  correspond  with  the  parameter  set  used  by  Merfeld  and  Zupan  (2002). 

G-excess  Tools 

In  hypergravity,  the  excess  G-forces  acting  on  the  otolith  organs  can  produce  sensations 
of  tilt  over-estimation,  tumbling  and  disorientation.  These  “G-excess”  phenomenon  are 
particularly  important  in  aviation,  where  changes  in  the  magnitude  and  direction  of  the  GIF 
vector  are  frequent,  abrupt,  and  often  occur  during  mission  critical  tasks  such  as  air  combat 
(Gilson,  Guedry,  Hixson,  &  Niven,  1973;  Guedry  &  Rupert,  1991).  The  Perception  Toolbox 
provides  a  separate  tool  for  visualizing  G-excess  effects. 

The  G-excess  Tools  GUI  is  shown  in  Figure  16.  G-excess  Tools  allows  users  to  modify 
the  relative  sensitivity  or  weighting  of  the  utricle  versus  saccule  organs  in  hypergravity  and  re¬ 
simulate  motion  profiles  with  the  adjusted  parameter  set.4  Estimates  of  roll  and  pitch  angle  can 
be  reviewed  and  animated  in  real  time  against  the  simulated  subject’s  true  orientation.  A 
dynamic  representation  of  the  forces  acting  on  the  otoliths  in  both  hyper-  and  normal-G 
environments  is  also  provided. 


4  This  feature  allows  modelers  to  distinguish  different  planes  of  motion  (Clark,  2013),  where  previously  they  had  to 
assume  all  planes  of  motion  were  weighted  the  same. 
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Figure  16.  G-excess  Tools  graphical  user  interface. 

Summary  List  of  Motion  Stimuli 

Table  6  lists  the  19  motion  profiles  incorporated  into  the  Perception  Toolbox  and  for 
which  model  findings  are  described  in  this  report. 

Table  6.  List  of  Motion  Profiles  Included  With  Perception  Toolbox 


1 

Constant  Velocity  Rotation  in  the  Dark 

2 

Constant  Velocity  Rotation  in  the  Light 

3 

Yaw  Circular  Vection 

4 

Fixed  Radius  Centrifugation 

5 

Post-Rotational  Tilt 

6 

Coriolis  during  Accelerated  Rotation 

7 

Coriolis  during  Constant  Velocity  Rotation 

8 

Coriolis  during  Decelerated  Rotation 

9 

G-Excess  Illusion 

10 

Somatogravic  Illusion  in  the  Dark 

11 

Somatogravic  Illusion  in  the  Light 

12 

Optokinetic  Nystagmus  &  After-Nystagmus 

13 

Linear  Vection 

14 

Roll  Circular  Vection 

15 

Post-Turn  Illusion  in  Degraded  Visual  Conditions 

16 

Post-Turn  Illusion  w/  Artificial  Visual  Horizon  Cue 

17 

Coriolis  Head  Movement  in  a  Coordinated  Turn 

18 

FI 8  Mishap  with  Angular  Cues 

19 

FI 8  Mishap  with  Angular  Cues  and  Acceleration  Cues 
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Sign  Conventions  and  Plot  Legends 


The  plot  legend  (x,  y,  z,  roll,  pitch,  yaw)  and  sign  conventions  used  for  all  simulations  in 
this  report  are  shown  in  Table  7  and  Table  8,  respectively.  For  each  simulation  presented,  the 
sensory  information  available  to  the  simulated  subject  is  summarized  in  a  Table  similar  to  the 
one  shown  in  Table  9.  The  two  vestibular  cues  (VEST)  correspond  to  angular  velocity  from  the 
semicircular  canals  (<n)  and  linear  acceleration  from  the  otolith  organs  (a).  The  four  visual  cues 
(VISUAL)  correspond  to  visual  position  (x),  visual  linear  velocity  (v),  visual  angular 
velocity  (d>),  and  visual  orientation  or  gravity  (g).  Cues  highlighted  in  yellow  □  are  available  to 
the  subject  during  each  simulation.  Cues  highlighted  in  red  ■  are  available  to  the  subject  during 
one  of  the  experimental  conditions  of  the  simulation  (see  captions  for  further  disambiguation). 
Cues  in  white  □  represent  sensory  information  that  are  not  used  or  available  (i.e.,  the  eyes  are 
closed,  the  room  is  darkened,  or  a  clear  visual  horizon  line  is  not  present). 

Table  7.  Plot  Trace  Color  Legend 


—  X,  ROLL 


—  Y,  PITCH 


—  Z,  YAW 


Table  8.  Sign  Conventions 


+  YAW 
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Table  9.  Sensory  Cues 
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Results  &  Discussion 


Basic  Sensory  Paradigms 

Earth  vertical  rotation  (dark,  light). 
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Figure  17  shows  the  model  response  for  constant  velocity  rotation  about  an  Earth  vertical 
axis  under  dark  (i.e.,  vestibular)  and  light  (i.e.,  visual-vestibular)  experimental  conditions. 
Velocity  perception  in  the  dark  (Figure  17A  and  17C)  displays  the  typical  exponential  decay  of 
angular  sensation  and  the  lengthened  time  constant  attributed  to  central  velocity  storage.  Since 
visual  information  is  not  present  in  this  condition,  the  simulation  results  are  similar  to  Merfeld  et 
al.’s  (1993)  ID  velocity  storage  model.  When  a  stationary  visual  surround  is  present  (Figure  17B 
and  17D),  the  model  predicts  a  sustained  sensation  of  rotational  motion  which  slightly  decays  to 
a  value  close  to  the  actual  input  stimuli. 
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Figure  1 7.  Model  predictions  for  constant  velocity  rotation  about  an  Earth- vertical  axis.  The 
simulated  subject  was  seated  upright  in  a  rotary  chair  and  rotated  in  the  dark  (A,  C)  and  light  (B, 
D)  at  =  0.26  radian/s  (14.89°/s)  for  a  duration  of  30  s.  (A,  B):  Estimated  yaw  angular  velocity  for 
darkness  and  lighted  conditions.  (C,  D):  Estimated  yaw  angle  for  darkness  and  lighted 
conditions.  The  Rotation  in  the  light  response  was  tuned  to  mimic  the  modeling  results  of  Borah 
et  al.  (1978)  and  the  experimental  data  of  Waespe  and  Henn  (1977).  Also  shown  is  the  input  0.26 
radian/s  stimulus  and  subsequent  yaw  angle  during  rotation. 
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Earth  vertical  rotation  (circular  vection). 
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In  response  to  pure  yaw  rotation  of  a  moving  visual  surround,  the  model  predicts  an 
illusory  sensation  of  angular  velocity  in  the  opposite  direction  of  the  visual  scene  (i.e.,  circular 
vection)  (Figure  18A  and  18B).  The  circular  vection  response  curve  shows  two  distinct 
components  associated  with  the  time  course  of  the  perceived  self-motion;  a  fast-rising 
component  responsible  for  the  quick  initial  onset  followed  by  a  slow  rising  component  that  levels 
out  to  a  value  slightly  below  the  velocity  of  the  visual  field.  The  fast-rising  component  accounts 
for  70%  of  the  total  angular  velocity  estimate  (0°/s  ->  10°/s  in  2.5  s)  and  is  driven  by  the  visual 
system  and  the  dynamics  of  the  visual  velocity  residual  feedback  loop.  As  the  internal  model’s 
estimate  of  angular  velocity  increases,  visual-vestibular  interactions  begin  to  slow  down  the 
neurological  processing  of  rotational  motion.  These  more  gradual  dynamics  result  from  the 
velocity  storage  time  constant  associated  with  the  canals  and  CNS,  and  account  for  the  remaining 
30%  of  velocity  perception  (10°/s  ->  14.5%  in  37.75  s).  The  existence  of  these  separate 
components  has  been  demonstrated  experimentally  (Cohen,  Henn,  Raphan,  &  Dennett,  1981; 

Jell,  Ireland,  &  Lafortune,  1984;  Lafortune,  Ireland,  Jell,  &  DuVal,  1986). 


Time  (sec) 


Time  (sec) 


Figure  18.  Model  predictions  for  constant  velocity  yaw  circular  vection.  The  subject  is  stationary 
and  placed  inside  an  optokinetic  drum  that  rotates  in  the  light  at  =  -0.26  radian/s  (14.89%). 

Drum  rotation  is  in  the  opposite  direction  of  the  angular  velocity  stimulus  to  illicit  an  illusory 
sensation  of  rotation  which  has  a  consistent  direction  with  the  angular  motion  in  the  Light/Dark 
example.  (A):  Estimated  yaw  angular  velocity.  (B):  Estimated  yaw  angle.  The  circular  vection 
response  was  tuned  to  mimic  the  modeling  results  of  Borah  (1978)  and  the  experimental  data  of 
Waespe  and  Henn  (1977).  Also  shown  is  the  input  -0.26  radian/s  drum  rotation  and  the  yaw 
angle  of  the  drum. 

Although  the  overall  circular  vection  simulation  is  good,  like  the  Borah  (1978)  (Figure  19 
curve  labeled  “CV”),  Pommellet  (1990),  and  Kynor  (2002)  models,  the  OMS  predicts  an 
immediate  onset  of  circular  vection  sensation  at  the  start  of  visual  field  motion.  This  is 
inconsistent  with  the  delay  typically  observed  in  human  subjects  (Lawson,  2005).  To  account  for 
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vection  delays  and  inter-subject  variability,  Borah  implemented  a  nonlinear  ad-hoc  conflict 
mechanism  that  could  distinguish  and  schedule  gains  for  pure  rotational  field  motion.  A  similar 
ad-hoc  augmentation  is  being  considered  for  future  modifications  of  our  model. 


Figure  19.  Borah,  Young,  and  Curry  (1978)  estimated  angular  velocity  for  Earth  vertical 
constant  velocity  rotation  in  the  dark  and  light  and  with  a  moving  visual  field  (circular  vection). 

Fixed  radius  centrifugation. 
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During  fixed  radius  centrifugation  the  CNS  must  dissociate  between  conflicting  cues 
from  the  semicircular  canals  and  otoliths.  The  shifting  gravitoinertial  force  vector  along  the  body 
produces  an  illusory  sensation  of  body  tilt  that  contrasts  to  the  pure  yaw  rotational  stimulation  of 
the  horizontal  semicircular  canals  (i.e.,  true  body  tilt  would  stimulate  the  roll  or  pitch  canal). 

Data  from  both  perceptual  reports  and  eye  movement  recordings  (Clark  &  Graybiel,  1963;  Clark 
&  Graybiel,  1966;  Graybiel  &  Brown,  1951;  Haslwanter,  Curthoys,  Black,  Topple,  &  Halmagyi, 
1996;  Merfeld,  Zupan,  &  Gifford,  2001;  McGrath,  Oman,  Guedry,  &  Rupert,  1993)  suggest  that 
the  illusory  tilt  will  exhibit  two  distinct  characteristics  during  acceleration  and  deceleration  of  the 
centrifuge.  During  acceleration  the  illusory  tilt  angle  will  lag  well  behind  the  actual  GIF  vector, 
slowly  building  up  to  a  steady  state  value  with  a  time  constant  that  varies  depending  on  the  GIF 
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level  and  onset  rate  of  the  centrifuge.  In  contrast,  during  deceleration  from  elevated  G,  little  or 
no  lag  is  typically  reported.  Additionally,  a  vertical  component  of  the  VOR  has  been  shown  to 
persist  during  centrifugation,  gradually  building  to  a  constant  value  as  the  centrifuge  reaches  a 
steady-state  velocity. 

Model  results  (Figure  20)  appear  to  match  these  findings.  The  inter-aural  (y-axis)  linear 
acceleration  stimulus  (Figure  20C)  and  estimated  response  (Figure  20D)  show  a  rapid  rise  as  the 
angular  velocity  (Figure  20A)  of  the  centrifuge  increases.  The  perceived  tilt  angle  (represented 
by  the  estimated  direction  of  the  gravitational  vector  in  Figure  20F),  however,  lags  considerably 
behind  the  acceleration  onset,  reaching  a  steady  state  in  approximately  60  s  (opposed  to  the 
actual  acceleration  onset  time  of  5  s).  Therefore,  this  aspect  of  our  model  requires  refinement. 
During  deceleration  (from  time  t  =  135  s  to  140  s)  no  such  lag  is  evident.  The  model  also  predicts 
the  expected  steady-state,  vertical,  slow  phase  eye  velocity  component  of  the  VOR  (Figure  20H) 
when  the  centrifuge  has  reached  a  constant  angular  rate. 


This  space  is  intentionally  blank. 
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Figure  20.  Model  response  during  175°/s  rotation  in  a  1-m  radius  centrifuge. 
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The  subject  was  situated  upright,  facing  away  from  the  direction  of  rotation,  and  aligned 
such  that  the  resultant  centripetal  acceleration  during  centrifugation  was  pointed  radially  inward 
(-y)  towards  the  axis  of  rotation.  Figure  21  shows  Actual  (A)  and  Estimated  (B)  angular  velocity, 
as  well  as  Actual  (C)  and  Estimated  (D)  linear  acceleration.  To  allow  for  a  direct  comparison 
with  the  modeling  results  of  Merfeld  and  Zupan  (2002)  and  the  published  data  of  Merfeld, 

Zupan,  and  Gifford  (2001),  the  simulated  conditions  precisely  match  the  175°/s  fixed-radius 
centrifuge  trials.  Also  shown  in  Figure  21  are  the  Actual  (E)  and  Estimated  (F)  gravitational 
vector,  and  the  TVOR  (G)  and  total  (H)  VOR.  For  the  TVOR  and  total  VOR,  an  eye  movement 
model  was  implemented  with  a  fixation  distance  (2  m)  and  leaky  integration  time  constant  (0. 1 
s).  These  values  are  identical  to  those  used  by  Merfeld  and  Zupan  (2002). 

These  results  are  in  agreement  with  the  data  cited  above  and  the  modeling  results  of 
Merfeld  and  Zupan  (2002)  (Figure  21).  As  noted  previously,  the  vestibular  core  of  the  OMS  has 
been  adapted  from  the  Observer  Model  developed  by  Merfeld  et  al.  (1993).  Agreement  between 
these  two  efforts  is  therefore  expected.  We  present  this  motion  paradigm  and  the  post-rotational 
tilt  paradigm  in  the  following  section  (“Post-rotational  tilt”)  to  demonstrate  that  the  new 
parameter  set  chosen  for  the  visual-vestibular  interaction  model  is  robust,  and  can  reproduce  both 
the  novel  simulations  considered  in  this  report  and  the  previously  validated  sensory  paradigms 
considered  in  past  modeling  efforts. 
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Figure  21.  Merfeld  and  Zupan  (2002)  modeling  results  for  fixed  radius  (1  m)  centrifugation.  The 
angular  velocity  input  (A)  is  shown,  along  with  the  model  estimate  of  angular  velocity  (B), 
gravity  (C),  and  linear  acceleration  (D),  and  predicted  the  translational  (E)  and  total  (F)  VOR. 
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Angular  Velocity  (%)  Estimated  Angular  Velocity  (%) 


Post-rotational  tilt. 
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Head  or  body  tilt  following  prolonged  constant  velocity  rotation  has  been  shown  to 
produce  dramatic  effects  on  the  post-rotatory  tilt  perception  and  VOR  response.  Benson  and 
colleagues  (Benson,  Bodin,  &  Bodin,  1966;  Benson,  1966a;  Benson,  1966b)  demonstrated  that 
the  time  constant  of  the  post-rotatory  VOR  is  reduced  in  response  to  a  change  in  the  static  otolith 
representation  of  gravity.  Merfeld,  Zupan,  and  Peterka  (1999)  and  Zupan,  Peterka,  and  Merfeld 
(2000)  also  demonstrated  that  the  illusory  tilt  that  results  from  post-rotatory  tilt  stimulation  will 
produce  a  small  central  estimate  of  linear  acceleration,  and  a  measureable  translational  VOR. 
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Figure  22.  Model  response  to  a  90°  post-rotational  tilt  (nose-down)  following  100°/s  constant 
velocity  Earth  vertical  yaw  rotation. 

The  yaw  angular  velocity  trapezoid  profile  (acceleration/deceleration  rates  of  50  °/s2)  and 
pitch  down  angular  velocity  spike  (peak  pitch  rate  of  90°/s)  are  shown  in  pane  A  (green  and  blue 
trace,  respectively).  Figure  22  shows  Actual  (A)  and  Estimated  (B)  angular  velocity,  Actual  (C) 
and  Estimated  (D)  linear  acceleration,  and  Translational  (E)  and  total  (F)  VOR.  For  the  TVOR 
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and  total  VOR,  an  eye  movement  model  was  implemented  with  a  fixation  distance  (2  m)  and 
leaky  integration  time  constant  (0. 1  s).  These  values  are  identical  to  those  used  by  Merfeld  and 
Zupan  (2002).  To  allow  for  a  direct  comparison  with  the  modeling  results  of  Merfeld  and  Zupan 
(2002)  and  the  published  data  of  Zupan  et  al.  (2000),  the  simulated  conditions  precisely  match 
the  100°/s  post-rotational  tilt  trials. 

Modeling  predictions  are  in  agreement  with  these  findings  (Figure  22).  The  predicted 
post-rotatory  time  constant  for  the  VOR  (Figure  22F)  of  the  simulated  trail  is  shown  to  be 
considerably  shorter  than  the  VOR  time  constant  during  initial  rotation  (5  s  vs.  27  s, 
respectively).  Likewise,  the  model  predicts  a  small  illusory  linear  acceleration  response  (Figure 
22D)  following  the  post-rotatory  tilt  and  a  subsequent  translational  VOR  (Figure  22E).  The 
modeling  results  of  Merfeld  and  Zupan  (2002)  are  reproduced  below  for  comparison  (Figure  23). 
Overall,  considerable  agreement  between  the  two  models  has  been  shown  for  both  the  fixed- 
radius  centrifuge  and  post-rotational  tilt  motion  paradigms. 


Figure  23.  Merfeld  and  Zupan  (2002)  modeling  results  for  a  90°  post-rotational  tilt  (nose-down). 
The  angular  velocity  input  (A)  is  shown,  along  with  the  model  estimate  of  angular  velocity  (B), 
gravity  (C),  and  linear  acceleration  (D),  and  predicted  the  translational  (E)  and  total  (F)  VOR. 
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Novel  Applications 

Guedry  and  Benson  (1978)  demonstrated  that  the  nauseogenicity  and  tumbling  sensations 
that  result  from  a  Coriolis  cross-coupled  head  movement  are  different  depending  on  the 
orientation  and  magnitude  of  the  resultant  angular  velocity  impulse  acting  on  the  semicircular 
canals.  In  this  classic  experiment,  volunteer  subjects  made  head  movements  during  three  distinct 
experimental  conditions:  accelerated  rotation,  constant  velocity  rotation,  and  decelerated 
rotation.  Each  head  movement  was  made  at  a  rotary  chair  velocity  of  1  rad/s.  Using  vector 
analysis,  Guedry  and  Benson  (1978)  proposed  a  simple  model  to  understand  why  these  three 
seemingly  identical  head  movements  produced  such  drastically  different  perceptual  responses. 
Input  for  the  simulations  are  shown  in  sections  “Coriolis  cross-coupling  during  accelerated 
rotation,”  “Coriolis  cross-coupling  during  constant  velocity  rotation,”  and  “Coriolis  cross¬ 
coupling  during  decelerated  rotation.”  They  were  chosen  to  match  each  of  the  three 
experimental  conditions  considered  by  Guedry  and  Benson  (1978). 

Coriolis  cross-coupling  during  accelerated  rotation. 
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The  predicted  response  to  a  cross-coupled  head  movement  made  during  accelerated 
rotation  is  shown  in  Figure  24.  Comparing  the  actual  (Figure  24A)  and  estimated  (Figure  24B) 
angular  velocity  response  shows  a  near-perfect  estimation  of  head  velocity  both  before  and  after 
the  30°  head  tilt.  As  the  head  tilt  is  initiated,  the  horizontal  semicircular  canal  cues  have  not  had 
time  to  substantially  decay  and  therefore  the  angular  velocity  vector  is  rotated  properly  into  the 
new  head  orientation  with  minimal  transient  illusory  linear  acceleration  (Figure  24D). 
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Figure  24.  Model  response  to  a  30°  head  tilt  during  Earth  vertical  accelerated  rotation.  Actual 
(A)  and  Estimated  (B)  angular  velocity.  Actual  (C)  and  Estimated  (D)  linear  acceleration. 


34 


The  simulated  subject  was  seated  upright  in  a  rotary  chair  and  accelerated  at  a  constant 
angular  rate  of  0.26  radian/s2  (14.89°/s2)  for  10  seconds.  At  3.85  seconds,  when  the  angular 
velocity  of  the  rotary  chair  was  equal  to  1  rad/s  (57.3°/s),  the  subject  made  a  30°  roll  head  tilt 
(Right  Ear  Down)  at  a  rate  of  60°/s  (red  trace).  The  simulation  was  modeled  to  match  the  classic 
Coriolis  cross-coupling  experiment  (Case  I)  performed  by  Guedry  and  Benson  (1978). 
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Figure  25.  Model  response  to  a  30°  head  tilt  during  accelerated  rotation.  Estimated  Roll  (A)  and 
Pitch  (B)  angle. 

As  is  seen  in  Figure  25,  proper  estimation  of  angular  velocity  and  linear  acceleration  will 
produce  a  correct  perception  of  roll  (Figure  25A)  and  pitch  (Figure  25B)  angle.  A  comparison 
between  Guedry  and  Benson’s  (1978)  vector  analysis  and  our  modeling  results  is  shown  in 
Figure  26.  The  vector  analysis  shows  the  y-  and  z-axis  angular  velocity  components,  and  the 
resultant  impulse  vector,  immediately  following  the  30°  heat  tilt. 


Figure  26.  Vector  analysis  of  estimated  angular  velocity  immediately  following  30°  roll  tilt.  (A): 
Predicted  response  from  Guedry  and  Benson  (1978).  (B):  Predicted  model  response  of  our  OMS. 
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Considerable  agreement  is  demonstrated  between  the  two  models,  in  terms  of  vector 
orientation  and  magnitude.  The  OMS  predicts  a  resultant  angular  velocity  impulse  vector  offset 
from  the  axis  of  chair  rotation  by  2.2°  (compared  with  0°  for  the  Guedry  and  Benson  [1978] 
model).  This  small  difference  in  offset  angle  would  be  expected  to  produce  minimal  illusory 
sensations  of  tilting/tumbling,  which  differ  negligibly  from  Guedry  and  Benson’s  (1978)  model, 
but  which  are  in  agreement  with  the  subjective  reports  of  motion  sickness  and  perceptual 
disturbances  recorded  during  the  original  experiment.  Some  vestibular  syndromes  have  highly 
predictable  effects  on  velocity  sensation,  which  should  be  amenable  to  modeling  (e.g.,  certain 
types  of  unilateral  vestibular  maladies)  and  targeted  therapies  or  prostheses. 

Coriolis  cross-coupling  during  constant  velocity  rotation. 
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The  predicted  response  to  a  cross-coupled  head  movement  made  during  constant  velocity 
rotation  is  shown  in  Figure  27.  In  this  simulation,  the  30°  head  tilt  is  made  when  angular  velocity 
perception  of  the  horizontal  semicircular  canal  has  effectively  decayed  to  zero  (Figure  27B).  As 
the  head  tilts  out  of  the  rotation  plane,  the  resultant  stimuli  to  the  superior  and  horizontal 
semicircular  canals  produce  a  response  traditionally  referred  to  as  the  Coriolis  cross-coupling 
illusion  or  the  vestibular  Coriolis  Effect.  The  result  is  an  illusory  sensation  of  angular  velocity 
and  tilt  about  a  third  axis  of  rotation;  this  illusion  can  be  highly  nauseogenic  when  a  gravity  cue 
is  present. 


This  space  is  intentionally  blank. 
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Figure  27.  Model  response  to  a  30°  head  tilt  during  Earth  vertical  constant  velocity  rotation.  The 
simulated  subject  was  seated  upright  in  a  rotary  chair  and  accelerated  at  a  constant  angular  rate 
of  0.26  radian/s2  (14.89°/s2)  to  a  terminal  velocity  of  1  rad/s  (57.3%).  At  60  s,  when  the 
horizontal  canal  cue  has  decayed  to  near  zero,  the  subject  made  a  30°  roll  head  tilt  (Right  Ear 
Down)  at  a  rate  of  60%  (red  trace).  The  simulation  was  modeled  to  match  the  classic  Coriolis 
cross-coupling  experiment  (Case  II)  performed  by  Guedry  and  Benson  (1978).  Actual  (A)  and 
Estimated  (B)  angular  velocity.  Actual  (C)  and  Estimated  (D)  linear  acceleration. 

During  accelerated  rotation,  the  angular  velocity  perception  immediately  preceding  the 
30°  tilt  was  nearly  veridical  and  the  cross-coupled  axis  was  aligned  properly  with  the  axis  of  the 
rotating  chair.  In  this  prolonged  constant- velocity  example,  the  cross-coupled  axis  is  not  aligned 
with  the  axis  of  chair  rotation  and  thus  illusory  sensations  of  tumbling  (Figure  27B),  acceleration 
(Figure  27D),  and  tilting  (Figure  27A  and  27B)  are  experienced. 
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Figure  28.  Model  response  to  a  30°  head  tilt  during  constant  velocity  rotation.  Estimated  Roll 
(A)  and  Pitch  (B)  angle. 

These  simulation  results  are  in  good  agreement  with  the  vector  analysis  of  Guedry  and 
Benson  (1978)  (Figure  29).  For  an  identical  simulation,  their  vector  analysis  predicted  an 
estimated  angular  velocity  magnitude  of  0.52  rad/s  with  individual  components  of  0.5  rad/s  and 
-0.13  rad/s  for  y-  and  z-axis  angular  velocity,  respectively.  This  corresponds  to  an  illusory  pitch 
down  sensation  about  an  axis  shifted  74.6°  from  the  true  vertical.  The  OMS  predicts  an  angular 
impulse  vector  of  magnitude  0.49  rad/s  with  0.48  rad/s  and  -0.1 1  rad/s  y-  and  z-axis  components, 
respectively.  The  OMS  also  predicts  that  the  axis  of  illusory  tumbling  will  be  shifted  72.9°  from 
the  true  vertical.  The  small  differences  between  our  modeling  results  and  Guedry’ s  theoretical 
calculations  can  be  attributed  to  the  0.5  s  latency  in  the  rolling  head  tilt.  Guedry  and  Benson 
(1978)  made  a  simplifying  assumption  that  the  tilt  was  accomplished  instantaneously,  whereas 
we  do  not. 


Figure  29.  Vector  analysis  of  estimated  angular  velocity  immediately  following  30°  roll  tilt.  (A): 
Predicted  response  from  Guedry  and  Benson  (1978).  (B):  Predicted  model  response. 
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The  Guedry  and  Benson  analysis  predicted  the  axis  of  tilt  sensation,  and  the  time  course 
of  angular  velocity  based  on  semicircular  canal  cues,  but  did  not  explicitly  quantify  the 
interaction  with  gravireceptor  cues,  or  predict  the  magnitude  and  duration  of  illusory  pitch 
(Figure  28B).  It  is  clear  from  subjects’  descriptions  that  the  perceived  pitch  angle  is  not  the 
integral  of  perceived  rotational  velocity  (Figure  27B).  Instead,  the  resulting  sensation  is  one  of 
continuous  tumbling,  but  limited  tilt.  The  OMS  is  able  to  successfully  mimic  this  paradoxical 
sensation. 

Coriolis  cross-coupling  during  decelerated  rotation. 
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During  decelerated  rotation,  reports  of  illusory  tumbling,  perceptual  disturbances,  and 
motion  sickness  were  graded  most  severe,  (as  compared  with  the  accelerated  and  constant 
velocity  conditions),  for  9  of  the  1 1  subjects  who  experienced  all  3  motion  profiles  (Guedry  & 
Benson,  1978).  The  remaining  two  subjects  reported  the  constant  velocity  and  deceleration 
condition  as  equally  nauseogenic  (Guedry  &  Benson,  1978).  Model  results  for  the  deceleration 
simulation  are  shown  in  Figure  30. 


C/3 

o- 


> 

1 

! 

=3 

< 


Figure  30.  Model  response  to  a  30°  head  tilt  during  Earth  vertical  decelerated  rotation.  Actual 
(A)  and  Estimated  (B)  angular  velocity.  Actual  (C)  and  Estimated  (D)  linear  acceleration. 
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The  simulated  subject  was  seated  upright  in  a  rotary  chair  and  accelerated  counter¬ 
clockwise  at  a  constant  angular  rate  of  0.26  radian/s2  (14.89°/s2)  to  a  terminal  velocity  of  2  rad/s 
(1 14.6°/s).  At  65  s  the  rotary  chair  was  decelerated  at  a  rate  of  -0.26  radian/s  (-14.89%  )  until  it 
was  brought  to  a  stop  at  72.7  s.  At  68.8  s,  when  the  angular  velocity  of  the  rotary  chair  was  equal 
to  1  rad/s  (57.3%),  the  subject  made  a  30°  roll  head  tilt  (Right  Ear  Down)  at  a  rate  of  60%  (red 
trace).  The  simulation  was  modeled  to  match  the  classic  Coriolis  cross-coupling  experiment 
(Case  III)  performed  by  Guedry  and  Benson  (1978). 


At  time  t  =  65  s,  as  the  chair  begins  to  decelerate  from  2  rad/s,  the  angular  velocity 
sensation  of  the  horizontal  semicircular  canal  has  effectively  decayed  to  zero  (Figure  30B).  The 
deceleration  of  the  chair  is  therefore  interpreted  as  angular  velocity  in  the  opposite  direction  of 
true  chair  motion.  At  the  moment  before  the  30°  head  tilt,  the  central  estimate  of  angular  velocity 
is  equal  to  -54.6%  vs.  the  true  angular  velocity  of  57.2%.  As  was  seen  in  the  constant  velocity 
simulation,  this  misestimation  results  in  an  impulse  vector  that  produces  a  strong  Coriolis  cross¬ 
coupling  illusion  and  illusory  sensations  of  tumbling  (Figure  30B),  acceleration  (Figure  30D), 
and  tilting  (Figure  31 A  and  3 IB). 
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Figure  31.  Model  response  to  a  30°  head  tilt  during  decelerated  rotation.  Estimated  Roll  (A)  and 
Pitch  (B)  angle. 


Once  again,  our  simulation  results  are  in  good  agreement  with  the  data  and  analysis  of 
Guedry  and  Benson  (1978)  (Figure  32A).  The  OMS  predicts  an  angular  impulse  vector  of 
magnitude  1.23  rad/s  (vs.  1.24  rad/s)  and  an  axis  of  illusory  tumbling  shifted  130.9°  (vs.  126.1°) 
from  the  true  vertical  (Figure  33B).  The  magnitude  of  the  angular  velocity  impulse  vector  during 
deceleration  (Figure  30B)  is  much  greater  than  the  vector  that  was  produced  during  the  constant 
velocity  scenario  (Figure  27B).  The  magnitude  of  the  tumbling  sensation  and  the  degree  of  axis 
shift  from  true  vertical  correctly  predict  that  the  deceleration  condition  produces  the  most 
perceptual  disturbances  and  instances  of  nausea  and  discomfort. 
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Figure  32.  Vector  analysis  of  estimated  angular  velocity  immediately  following  30°  roll  tilt.  (A): 
Predicted  response  from  Guedry  and  Benson  (1978).  (B):  Present  model  response. 

G-Excess  illusion. 
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When  a  pilot  makes  a  head  tilt  in  hypergravity,  the  excess  gravitoinertial  force  acting  on 
the  otolith  organs  can  produce  non-veridical  sensations  of  aircraft  banking  and  spatial 
disorientation  (Figure  33).  This  phenomenon  is  typically  referred  to  as  the  G-Excess  illusion  or 
G-Excess  Tilt  Illusion. 


This  section  is  intentionally  blank. 


41 


0.5  G 


Figure  33.  Gravitoinertial  force  projection  following  a  head  turn  made  in  normal  (A,  B,  C)  and 
hypergravity  (D,  E,  F)  conditions.  (A):  Head  upright  at  1G.  (B):  Tilting  the  head  30°  will  result 
in  a  0.5G  shear  force  acting  alone  the  inter-aural  utricular  plane  of  the  otoliths.  (C):  At  90°  the 
utricular  plane  is  aligned  with  gravity  and  a  1.0G  shear  force  is  experienced.  (D):  Head  upright  at 
2G.  (E):  Tilting  the  head  30°  will  now  result  in  a  1.0G  inter-aural  shear  force  (2.0G  sin  30°  = 
LOG)  which  is  identical  to  force  experienced  during  the  90°  head  tilt  at  LOG  (C).  (F):  In  the 
absence  of  strong  visual  or  proprioceptive  cues,  this  elicits  an  illusory  overestimation  of  head  tilt 
of  up  to  90°  (commonly  referred  to  as  the  G-Excess  illusion),  which  is  partially  generalized  to 
one’s  body,  seat,  and  vehicle.  (Reproduced  from  Clark  [2013].) 

Experiments  investigating  the  G-Excess  illusion  have  used  centrifugation  (Chelette, 
Martin,  &  Albery,  1995;  Schone,  1964;  Correia,  Hixson,  &  Niven,  1968;  Miller  &  Graybiel, 
1971)  to  produce  G  forces.  While  this  method  is  appropriate  to  study  this  phenomenon,  it  relies 
on  strong  angular  velocity  stimulation  of  the  semicircular  canals  to  produce  the  required  hyper- 
gravic  environment.  The  resulting  illusion  is  a  combination  of  both  G-excess  and  Coriolis  cross¬ 
coupling,  which  makes  modeling  and  understanding  the  dynamics  of  the  pure  G-Excess  illusion 
very  difficult. 

Coordinated  aircraft  turning  maneuvers  have  been  employed  to  reduce  semicircular  canal 
contributions  greatly  (Gilson,  Guedry,  Hixson,  &  Niven,  1973;  Guedry  &  Rupert,  1991).  To 
dissociate  between  these  two  components  (angular  versus  linear),  we  instead  model  G-excess 
during  sustained,  purely  vertical  linear  acceleration  (Figure  34).  The  simulated  subject  would  be 
seated  upright  in  a  darkened,  very  tall  elevator  shaft  and  accelerated  upwards  at  2  Gs  for  25  s 
(Figure  34A).  The  resulting  gravitoinertial  force  acting  along  the  long  body  (z)  axis  is  equal  to  - 
3.0  Gz  (f  =  g-a  =  -lG-2G  =  -3  Gz).  At  10  s,  the  simulated  subject  makes  a  smooth  45°  right- 
ear-down  rolling  head  movement  (head  angular  velocity  profile  shown  in  Figure  34C).  This  head 
orientation  is  held  for  7  s  and  then  returned  back  to  the  neutral  spine  position. 
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Figure  34.  Model  response  to  a  45°  head  tilt  during  2G  vertical  linear  acceleration.  Actual  (A) 
and  Estimated  (B)  linear  acceleration.  Actual  (C)  and  Estimated  (D)  angular  velocity. 

The  model  predicts  that  the  subject  will  estimate  both  components  of  the  linear 
acceleration  force  acting  along  the  body  (Figure  34B)  and  the  roll  angular  velocity  of  the  head 
with  considerable  accuracy  (Figure  34D).  While  the  individual  components  of  the  linear 
acceleration  vector  (Figure  34B)  are  slightly  under-reported  in  terms  of  magnitude  (the  2.0  Gz 
sustained  upward  acceleration  is  estimated  as  1.6  Gz),  the  relative  orientation  of  the  vector  is 
precisely  aligned  with  the  true  orientation  of  the  head.  The  resulting  perception  of  roll  angle  is 
therefore  accurate  (Figure  34A)  and  no  G-Excess  illusion  is  experienced.  This  is  not  concordant 
with  the  literature. 

In  order  to  produce  the  overestimation  of  roll  angle  that  we  would  expect  (Gilson  et  al., 
1973;  Guedry  &  Rupert,  1991)  during  this  hyper-G  simulation,  we  must  modify  an  internal 
parameter  of  the  vestibular  system  model.  The  original  model,  based  on  Merfeld  et  al.’s  (1993) 
Observer  structure,  was  developed,  tested  and  verified  strictly  under  1-G  normal  gravity 
conditions.  That  model  implicitly  assumes  that  the  central  nervous  system  has  an  internal 
representation  of  gravity  and  can  distinguish  tilt  from  translation  with  equal  ability  in  all 
gravitoinertial  force  conditions.  Furthermore,  the  model  assumes  that  the  utricle  and  saccule 
otolith  feedback  parameters  are  equally  weighted  during  deconstruction  of  the  estimated  linear 
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acceleration  vector.  In  other  words,  as  the  head  is  tilted  45°  during  our  vertical  acceleration 
simulation,  the  CNS,  by  design,  properly  extracts  the  vector  components  of  the  GIF  that 
represent  gravity  (due  primarily  to  correct  estimation  of  the  angular  velocity  of  the  head  by  the 
canals)  and  interprets  the  remaining  GIF  as  acceleration  in  the  proper  direction  of  motion. 
Significant  overestimation  of  tilt  angle  is  therefore  impossible  with  the  original  Merfeld  model. 
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Figure  35.  Model  response  to  a  45°  head  tilt  during  2G  vertical  linear  acceleration.  (A): 
Estimated  roll  angle.  (B):  Estimated  pitch  angle.  The  true  roll  and  pitch  angle  are  also  shown. 


We  extended  the  original  vestibular  model  to  process  utricle  and  saccule  shearing  forces 
differently  by  simply  modifying  the  Kaa  parameter  of  the  CNS  model.  This  approach  was 
originally  suggested  by  Clark  (2013)  and  has  been  implemented  here  using  a  simplified  version 
of  his  approach.  For  all  of  the  present  simulations  Kaa  was  set  to  -4  for  feedback  along  the  x- 
axis,  y-axis,  and  z-axis.  Errors,  therefore,  in  estimation  of  the  individual  components  of  linear 
acceleration  were  equally  weighted.  If  the  x-  and  y-components  of  Kaa  are  instead  set  to  -2 
during  elevated  G  simulations,  and  the  z-component  is  left  at  -4,  misestimation  of  tilt  is  possible. 
Modifying  Kaa  implies  that  the  CNS  will  weight  utricle  and  saccule  information  differently 
when  estimating  gravity  and  acceleration  in  hyper-G  and  will  allow  shifts  in  the  relative 
orientation  of  the  central  linear  acceleration  estimate  to  affect  the  resultant  GIF  direction 
independently  from  the  central  estimate  of  gravity  (or  tilt  angle). 


Results  for  model  simulations  using  this  modified  parameter  set  are  shown  in  Figures  21 
and  36.  The  experimental  setup  for  this  simulation  is  identical  to  the  previous  example.  Notice 
that  angular  velocity  perception  (Figure  36D)  remains  veridical.  However,  due  to  the 
modification  of  Kaa,  the  central  estimate  of  linear  acceleration  (Figure  36B)  now  produces  a 
large  overestimation  of  roll  angle  (Figure  37A).  The  model  predicts  an  initial  G-excess 
overestimation  of  tilt  angle  of  15.6°,  and  this  misperception  slowly  worsens  to  a  25.6° 
overestimation  as  the  head  turn  is  held  under  G.  The  model  also  predicts  a  lag  in  tilt  angle 
estimation  that  lasts  for  4  s  following  the  head  turn  back  to  neutral  spine  orientation. 
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Figure  36.  Model  response  to  a  45°  head  tilt  during  2G  vertical  linear  acceleration  with  modified 
Kaa  parameter.  Actual  (A)  and  Estimated  (B)  linear  acceleration.  Actual  (C)  and  Estimated  (D) 
angular  velocity. 
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Figure  37.  Model  response  to  a  45°  head  tilt  during  2G  vertical  linear  acceleration  with  modified 
Kaa  parameter.  (A):  Estimated  roll  angle.  (B):  Estimated  pitch  angle.  The  true  roll  and  pitch 
angle  are  also  shown. 
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Somatogravic  illusion  (dark,  light). 
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The  model  successfully  predicts  the  somatogravic  (pitch-up)  illusion  for  forward  linear 
acceleration  in  the  dark  (Figures  38A  and  38B)  and  the  suppression  of  the  illusion  in  the  light 
(Figures  38C  and  38D).  The  somatogravic  illusion  in  the  dark  has  been  well-documented 
experimentally  (Cohen,  Crosby,  &  Blackburn,  1973;  Graybiel  &  Brown,  1951;  Graybiel,  1966) 
and  is  predicted  by  many  perception  models  (Borah  et  ah,  1978;  Pommellet,  1990;  Kynor,  2002; 
Bilien,  1993).  It  is  worth  comparing  the  implementation  of  the  visual  pathways  for  these  models 
in  order  to  understand  how  the  OMS  works  differently  than  previous  modeling  efforts. 


Figure  38.  Model  response  to  a  step  in  forward  linear  acceleration.  The  simulated  subject  was 
seated  upright  and  accelerated  forward  (+x)  on  a  horizontal  sled  at  0.2  G/s  for  10  s  in  both 
darkness  (A,  B)  and  lighted  (C,  D)  conditions.  The  time  course  and  dynamics  of  the  predicted 
pitch-up  sensation  were  set  to  match  the  experimental  centrifuge  data  from  Graybiel  and  Brown 
(1951)  and  the  Borah  et  al.  (1978)  KF  response  curves.  (A,  C):  Estimated  linear  acceleration  for 
darkness  and  lighted  conditions.  (B,  D):  Estimated  pitch  angle  for  darkness  and  lighted 
conditions.  Also  shown  is  the  0.2  G  input  stimulus. 

The  OMS  uses  visual  angular  velocity  and  visual  gravity  information,  along  with  canal 
and  otolith  cues  from  the  vestibular  system,  to  estimate  acceleration  and  pitch  angle.  While 
visual  linear  velocity  and  position  information  are  also  available,  the  structure  of  the  OMS  posits 
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that  these  quantities  do  not  appreciably  affect  the  lower  derivatives  of  acceleration  or  the  relative 
orientation  of  the  simulated  subject.  This  constrained  visual  capability  is  quite  different  from  past 
modeling  implementations,  which  did  not  incorporate  visual  cues  for  the  direction  of  gravity. 
These  models  assumed  that  only  visual  velocity  information,  resulting  from  linear  motion  of  the 
moving  scene,  would  suppress  the  somatogravic  illusion  and  permit  correct  estimates  of  forward 
linear  acceleration.  In  a  study  addressing  this  specific  issue,  Tokumaru  et  al.  (1998)  found  that 
subjects  who  underwent  linear  acceleration  with  isolated  visual  vection  cues  lacking  visual 
gravity  cues  reported  sensations  of  tilt  with  equal  likelihood  and  magnitude  as  those  without  any 
visual  information  at  all.  The  researchers  additionally  found  that  when  subjects  were  provided 
with  a  visual  gravity  reference  they  felt  significantly  reduced  magnitudes  of  the  pitch-up 
sensation.  These  findings  imply  that  while  many  models  properly  predict  visual  suppression  of 
the  somatogravic  illusion,  the  implementation  made  by  the  OMS  is  more  consistent  with  the 
findings  and  with  the  actual  visual-vestibular  interaction  mechanism  responsible. 

The  somatogravic  illusion  is  common  in  aviation;  it  is  particularly  hazardous  during 
flight  in  degraded  visual  conditions  and  during  rapid  ascent  or  catapult-assisted  take  off.  In  these 
conditions,  a  proper  representation  of  visual-vestibular  sensory  interaction  is  required  in  order  to 
predict  the  onset  and  intensity  of  the  resulting  pitch-up  sensation.  Unlike  previous  modeling 
efforts,  the  OMS  can  predict  the  somatogravic  illusion  in  light,  as  long  as  a  strong  visual 
orientation  cue  is  not  present  (e.g.,  a  clear  horizon  line). 


Optokinetic  nystagmus  (OKN)  &  optokinetic  after-nystagmus  (OKAN). 
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Results  for  a  simulation  of  optokinetic  nystagmus  (OKN)  and  optokinetic  after¬ 
nystagmus  (OKAN)  are  shown  in  Figure  39.  The  predicted  model  dynamics  for  OKN  and 
OKAN  (Figures  39D,  39E,  and  39F)  are  compared  against  data  from  Raphan  et  al.  (1979) 
(Figures  39A,  39B,  and  39C).  The  model  response  to  the  rotation  in  the  dark  (Figure  39D) 
demonstrates  the  expected  exponential  decay  of  slow  phase  velocity  during  constant  velocity 
rotation  and  the  post-rotatory  response  in  the  opposite  direction  as  the  chair  is  stopped  at  t  =  60  s. 
During  pure  rotation  of  the  visual  surround  (circular  vection),  the  model  correctly  predicts  the 
fast  and  slow  dynamics  of  the  optokinetic  nystagmus  onset  and  the  saturation  of  slow  phase  eye 
velocity  that  immediately  occurs  once  the  visual  vection  field  is  turned  off  (Figure  39E).  The 
model  estimates  a  rapid  saturation  of  velocity  to  about  90°/s,  which  is  slightly  lower  than  the 
120°/s  OKAN  saturation  level  reported  by  Raphan  et  al.  (1979).  Finally,  during  constant  velocity 
rotation  in  the  light  (Figures  39C  and  39F),  the  model  accurately  predicts  the  rapid  rise  and 
nearly  veridical  initial  estimate  of  slow  phase  eye  velocity,  the  slight  decline  in  perception  during 
constant  velocity  rotation,  and  the  small  negative  velocity  recorded  as  the  chair  is  brought  to  a 
stop. 
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Figure  39.  Slow  phase  eye  velocity  in  response  to  180°/s  rotation  in  the  dark  (A,  D),  circular 
vection  (B,  E)  and  rotation  in  the  light  (C,  F).  (A,  B,  C):  Experimental  data  from  Raphan  et  al. 
(1977).  (D,  E,  F):  Predicted  model  response.  Note:  Model  output  is  represented  as  stem  plots  to 
provide  a  better  visual  comparison  with  the  Raphan  et  al.  (1977)  data.  Slow  phase  eye  velocity 
was  calculated  with  an  eye  movement  model  that  used  a  fixation  distance  of  (2  m)  and  a  leaky 
integration  time  constant  of  (0.1  s).  These  values  are  identical  to  those  used  by  Merfeld  and 
Zupan  (2002). 
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Linear  vection. 
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In  this  simulation,  the  subject  sat  upright  and  viewed  a  high  fidelity  translating  visual 
scene.  Input  to  the  model  is  represented  as  ±15  cm/s  and  ±7  cm/s  steps  in  visual  linear  velocity 
(Figure  40 A).  As  the  CNS  gradually  accepts  the  visual  input,  an  illusory  sensation  of  linear  self- 
motion  (linear  vection)  develops  in  a  direction  opposite  to  visual  field  motion  (Figure  40B).  To 
allow  for  a  direct  comparison  with  Chu  (1976)  (Figure  40D),  we  plotted  the  negative  of  the 
visual  field  velocity  in  Figure  55A.  While  the  model  yields  useful  predictions  concerning  the 
Chu  (1976)  data,  as  with  the  circular  vection  simulation,  the  model  will  require  modification  to 
predict  vection  onset  delays  and  the  complex  dynamics  that  result  from  such  delays  during  step 
changes  in  the  visual  vection  field  (Figure  40E).  A  more  complicated  visual  system,  perhaps  one 
which  dissociates  focal  and  peripheral  vision  or  implements  nonlinear  visual  dynamics,  would  be 
required  to  produce  these  characteristics. 

Due  to  the  structure  of  the  visual  residual  pathways  assumed  in  the  OMS,  changes  in 
visual  flow  velocity  are  not  interpreted  as  acceleration,  and  do  not  influence  the  estimated 
direction  of  gravity  or  pitch  (Figure  40C).  This  is  why  in  this  example,  a  step  change  in  visual 
field  velocity  does  not  result  in  any  transient  tilt  sensation,  which  was  accurate.  As  was  noted 
during  the  somatogravic  illusion  simulation  section  “Somatogravic  illusion  (dark,  light),” 
previous  attempts  to  model  the  interactions  of  the  visual  and  vestibular  systems  did  not  account 
for  this  important  structural  distinction.  A  step  change  in  linear  field  velocity  therefore  could 
result  in  transient  acceleration  and  a  sensation  of  pitching  forward  or  backwards,  which  is  not 
observed  experimentally. 


This  space  is  intentionally  blank. 
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Figure  40.  Model  response  during  step  changes  in  vection  field  velocity.  To  allow  for  a  direct 
comparison  with  the  published  data  of  Chu  (1976),  the  simulated  conditions  precisely  match  the 
±  15  cm/s  and  ±  7.5  cm/s  linear  vection  steps  used  during  experimentation  (D).  (A):  Negative 
Vection  Field  Velocity  (-x).  (B):  Estimated  linear  velocity  (C):  Estimated  pitch  angle.  Vection 
field  velocity  (D)  and  subjective  linear  vection  reports  (E)  from  Chu  (1976). 
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Roll  circular  vection. 
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In  the  previous  two  simulations  of  circular  vection  (sections  “Earth  vertical  rotation 
[circular  vection]”  and  “Optokinetic  nystagmus  [OKN]  &  optokinetic  after-nystagmus 
[OKAN]”)  the  angular  velocity  vector  of  the  vection  field  was  aligned  with  gravity  and  produced 
only  illusory  self  motion  about  an  upright  yaw  axis.  For  both  of  these  examples,  the  CNS  was 
able  to  estimate  angular  velocity  without  interaction  or  stimulation  of  the  otolith  organs.  If  we 
shift  the  orientation  of  the  vection  field,  we  can  induce  self-motion  about  a  roll  or  pitch  body 
axis.  In  this  scenario  the  visual  system,  semicircular  canals,  and  otoliths  all  interact  to  form  an 
estimate  of  both  orientation  and  angular  self-motion.  During  circular  vection  about  a  roll  or  pitch 
body  axis,  a  paradoxical  sensation  of  limited  displacement  in  tilt  (Mast,  Berthoz,  &  Kosslyn, 
2001)  while  experiencing  a  continuous  tumbling  velocity  sensation  is  often  reported  (Allison, 
Howard,  &  Zacher,  1999).  This  sensation  is  commonly  described  as  velocity  without 
displacement  or  tumbling  without  getting  anywhere.  The  degree  of  tumbling  and  angle  of  tilt  are 
dependent  on  the  strength  of  the  visual  cue  and  the  size  of  the  visual  field. 


To  model  this  example,  we  position  a  simulated  subject  in  front  of  a  wide  field  of  view 
roll  vection  field  that  rotates  at  a  velocity  of  45°/s.  We  assume  that  the  subject  can  extract  a 
visual  angular  velocity  cue  from  the  vection  field  but  cannot  extract  a  strong  visual  orientation 
cue. 
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Figure  41.  Model  response  to  a  45°/s  circular  roll  vection  stimulus.  (A):  Estimated  roll  angular 
velocity.  (B):  Estimated  roll  angle.  The  drum  velocity  is  also  shown  in  pane  A. 

Results  (Figure  41)  show  that  the  model  is  able  to  successfully  predict  the  paradoxical 
sensations  that  are  typically  experienced  during  circular  roll  vection.  For  this  simulation,  the 
subject  estimates  a  steady  state  static  tilt  angle  of  -48°  (Figure  4 IB)  and  a  persistent  angular 
velocity,  or  tumbling  sensation,  at  a  rate  of  -36°/s  (Figure  41  A).  The  Perception  Toolbox 
provides  a  tool  to  visualize  this  paradoxical  sensation  (Figure  42).  During  the  simulation,  the 
actual  orientation  of  the  subject  remains  constant  (only  the  visual  scene  changes  orientation) 
(Figure  42A).  The  estimated  orientation,  however,  is  comprised  of  two  individual  sensations: 
static  tilt,  represented  by  the  solid  avatar  in  Figure  42B,  and  constant  tumbling,  represented  by 
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the  translucent  rotating  avatar.  When  viewed  in  real  time,  this  visualization  tool  provides  a 
powerful  illustration  of  the  combined  perceptual  response. 


Figure  42.  Perception  Toolbox  tilting  and  tumbling  visualization  tool  during  constant  velocity 
roll  circular  vection.  (A):  Actual  subject  orientation  throughout  simulation.  (B):  Screenshots  of 
the  animated  avatar  showing  the  constant  tilt  angle  (solid  avatar)  and  persistent  tumbling 
sensation  (translucent  avatar)  experienced. 

It  is  important  to  note  that  if  a  strong  visual  orientation  cue  is  present  during  simulation 
(i.e.,  a  polarized  floor  and  ceiling),  the  model  would  predict  a  360°  rotating  tilt  angle  that 
coincides  closely  with  the  rotation  of  the  visual  scene.  These  results  are  consistent  with  the 
research  of  Allison  et  al.  (1999),  who  demonstrated  that  almost  80%  of  subjects  positioned  in  a 
furnished  rotating  room  experienced  complete  360°  tumbling  sensations. 

Practical  Aviation  Applications 

Post-turn  illusion  in  degraded  visual  conditions. 
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The  passage  below,  reproduced  from  Lawrence  Young’s  chapter  of  the  2003  edition  of 
Principles  and  Practice  of  Aviation  Psychology,  provides  a  thorough  description  of  the  post-turn 
illusion. 

When  a  pilot  banks  the  airplane  into  a  right  wing  down  constant  rate  level  turn  in 
the  clouds,  the  information  is  registered  by  the  horizontal  semicircular  canals, 
which  register  the  initial  yaw  rate,  and  by  the  vertical  canals,  which  measure  the 
brief  roll  rate  on  entry  into  the  turn  [...].  Assuming  a  coordinated  turn  with  no 
sideslip,  the  otolith  organs  and  the  haptic  receptors  continue  to  register  the  net 
gravito-inertial  force  direction  along  the  z-axis  of  the  aircraft,  although  at  a 
magnitude  greater  than  1  G.  Several  seconds  into  the  turn,  the  horizontal 
semicircular  canals  signal  a  steadily  reduced  yaw  rate,  which  finally  drops  below 
subjective  threshold  at  a  time  determined  by  the  initial  turn  rate.  At  this  point,  in 
the  absence  of  any  confirming  out-the-window  visual  cues,  a  passenger  would 
feel  the  airplane  to  be  neither  banked  nor  turning,  but  flying  straight  and  level. 

And  so  would  the  pilot  if  no  reference  were  made  to  the  artificial  horizon  or  the 
turn  indicator.  When  the  pilot  then  rolls  back  left  to  straight  and  level  flight,  with 
the  GIF  still  directed  into  the  seat,  the  passenger's  vertical  semicircular  canals 
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correctly  detect  a  brief  roll  to  the  left.  The  horizontal  canals,  however,  which  had 
been  indicating  no  yaw  rate,  now  experience  a  sudden  change  in  angular  velocity 
to  the  left,  which  they  now  duly  signal  to  the  brain.  Pilot  and  passenger  alike  feel 
that  they  have  begun  a  left  turn.  (Young,  2003,  p.  84-86) 

In  the  next  two  sections  (“Post-turn  illusion  in  degraded  visual  conditions”  and  “Post-turn 
illusion  with  artificial  visual  horizon  orientation  cue”),  we  simulate  two  examples  of  the  post- 
turn  illusion  following  a  coordinated  two-minute-tum.  For  simplicity,  we  only  consider  the 
forces  that  are  produced  due  to  aircraft  rotation  and  centripetal  acceleration  and  do  not  attempt  to 
model  atmospheric  interactions,  lift,  drag,  or  other  more  complicated  aerodynamic  characteristics 
of  the  flight  profde.  These  factors  are  important  to  flight  control  but  are  not  important  to 
illustrate  this  particular  illusion. 

We  assume  that  the  pilot  is  situated  in  an  aircraft  that  rolls  about  the  center  of  the  head. 

At  time  t  =  0,  the  pilot  enters  a  coordinated  two-minute-tum  with  a  true  airspeed  of  120  knots 
and  a  subsequent  bank  angle  of  18.25°  (resultant  radius  of  the  turn  is  equal  to  0.6366  nautical 
miles).  A  2-min  turn  under  these  conditions  will  result  in  a  total  turn  velocity  of  3°/s  (360°  in  120 
seconds  =  3°/s).  As  the  aircraft  banks  into  the  coordinated  turn,  the  roll-  or  x-axis  semicircular 
canal  will  register  the  aircraft  bank  rate  (18.25°/s  for  this  example).  Simultaneously,  the  3°/s 
angular  velocity  of  the  turn  will  be  projected  along  the  y-  and  z-axes  of  the  semicircular  canals. 
The  net  GIF  vector  throughout  the  simulation  will  remain  directed  into  the  seat;  however,  the  y- 
and  z-axes  of  the  otolith  organs  will  be  stimulated  by  the  centripetal  acceleration  of  the  turn.  This 
acceleration  is  directed  toward  the  center  of  rotation.  Figure  43  summarizes  all  of  the  forces 
acting  on  the  semicircular  canals  and  otolith  organs  during  the  coordinated  2-min  turn. 
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Figure  43.  Model  input  to  the  semicircular  canal  (A)  and  otolith  organs  (B)  during  a  coordinated 
2-min  turn.  (A):  Semicircular  canal  angular  velocity  projection.  (B):  Otolith  linear  acceleration 
projection.  The  radius  of  curvature  of  the  coordinated  turn  was  equal  to  0.637  nautical  miles. 
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For  this  simulation,  we  assumed  that  the  pilot  was  flying  under  degraded  visual 
conditions  and  failed  to  look  (or  attend  to)  the  attitude  indicator  sufficiently  during  the  turning 
maneuver.  Under  these  conditions,  the  model  successfully  predicts  the  dynamics  of  the  post-turn 
illusion  as  described  by  Young  (Figure  44).  As  the  pilot  enters  the  turn,  he  initially  experiences  a 
sensation  of  roll  tilt  in  the  proper  direction  of  aircraft  bank.  This  cue  quickly  washes  out  and 
soon  the  pilot  feels  as  if  he  is  flying  straight  and  level.  After  2  min,  as  he  begins  to  roll  out  of  the 
turn,  he  perceives  that  he  is  banking  in  the  opposite  direction  (magnitude  of  illusory  bank  =18°) 
of  actual  aircraft  motion  and  entering  a  right  turn.  This  illusion  is  relevant  to  spatial 
disorientation  that  can  occur  during  “go  around”  maneuvers  following  a  missed  approach 
(McGrath,  Newman,  Lawson,  &  Rupert,  2015). 
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Figure  44.  Estimated  roll  angle  during  coordinated  turn  without  visual  sensory  input. 


Post-turn  illusion  with  artificial  visual  orientation  cue. 
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In  this  simulation,  we  repeat  the  coordinated  2-min  turning  profile  used  in  the  section 
“Post-turn  illusion  in  degraded  visual  conditions”  but  add  a  visual  orientation  cue  from  an 
artificial  horizon  indicator  such  as  an  attitude  display.  We  model  this  visual  input  as  a  unit  vector 
that  maintains  alignment  with  the  true  aircraft  bank  angle.  Perception  of  aircraft  bank  angle  is 
now  driven  by  the  rotation  of  the  aircraft  registered  by  the  semicircular  canals,  the  resultant 
acceleration  vectors  due  to  circular  motion  acting  on  the  otolith  organs,  and  a  visual  orientation 
vector  that  steers  the  central  estimate  of  bank  towards  the  true  orientation.  The  estimated  roll 
angle  of  the  aircraft  under  these  conditions  is  shown  in  Figure  45. 
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Figure  45.  Estimated  roll  angle  during  coordinated  turn  with  a  visual  orientation  cue  from  an 
artificial  horizon  indicator. 

The  magnitude  of  the  post-turn  illusion  is  drastically  reduced  with  a  visual  orientation 
cue.  As  the  pilot  rolls  out  of  the  coordinated  turn,  he  experiences  a  brief,  4.5°  banking  roll  tilt  in 
the  opposite  direction  of  the  original  turn.  This  corresponds  to  a  75%  reduction  (4.5°  vs.  18°)  in 
the  magnitude  of  the  post-turn  illusion  compared  with  the  simulation  under  degraded  visual 
conditions. 

Coriolis  head  movement  during  a  coordinated  turn. 


10 


-10 


VEST 

VISUAL 

co  a 

x  v  g  co 

For  the  final  example,  we  simulate  a  head  tilt  made  during  a  3.6  Gz  constant  rate  level 
turn.  At  250  knots,  the  bank  angle  for  the  turning  maneuver  is  33.4°.  Once  the  aircraft  has 
entered  the  coordinated  turn,  we  assume  that  the  pilot  makes  a  30°  rolling  head  movement  in  the 
opposite  direction  of  the  turn  to  approximately  align  the  head  with  the  true  horizon  line.  Such 
behavior  is  commonly  observed  in  flight  and  on  the  ground,  and  is  attributable  to  well-known 
visually-mediated  righting  reflexes  (Magnus,  1926;  Young  et  ah,  1986;  Merryman  &  Cacioppo, 
1997),  which  incorporate  visual  frame  of  reference  information  into  the  control  of  body  posture 
and  orientation  perception. 


Figure  46.  Angular  velocity  perception  following  a  cross-coupled  head  movement  during  a 
coordinated  turn.  Actual  (A)  and  estimated  (B)  angular  velocity. 
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The  angular  velocity  of  the  aircraft  and  the  estimated  angular  velocity  perceived  by  the 
pilot  are  shown  in  Figure  46 A  and  Figure  46B,  respectively.  A  vector  analysis  of  the  angular 
velocity  impulse  immediately  following  the  head  tilt  reveals  a  small  forward,  pitch  down, 
tumbling  velocity  of  magnitude  7.25°/s  (Figure  47).  This  small  angular  rate  is  unlikely  to  induce 
a  strong  sensation  of  tumbling,  vertigo,  or  spatial  disorientation.  An  increased  turn  rate  would 
intensify  the  cross-coupled  illusion.  Even  under  the  moderately  stressful  conditions  simulated  in 
this  example;  the  response  is  not  very  provocative.  We  would  expect  pilots  to  minimize  these 
types  of  head  movements  if  they  were  particularly  disorienting.  Angular  velocity  of  the  aircraft  is 
not  usually  the  reason  for  disturbance  during  head  movement  (Gilson  et  al.,  1973;  Guedry  & 
Rupert,  1991). 
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Figure  47.  Vector  analysis  of  estimated  angular  velocity  immediately  following  30°  roll  tilt. 

Note  that  the  tilted  orientation  of  the  pilot’s  head  is  offset  from  the  true  horizon  line  by 
only  2°.  The  resultant  magnitude  of  the  angular  velocity  tumbling  vector  (Figure  47)  (black 
arrow)  is  equal  to  7.257s  forward  (into  the  page),  and  the  estimated  illusory  sensation  of  pitch  is 
equal  to  approximately  4°  nose  down. 

Case  Study:  F18  Mishap  Analysis 

Mishap  summary. 

On  August  10th  2011,  two  U.S.  Marine  aviators  ejected  from  a  McDonnell  Douglas  F/A- 
18D  Hornet  during  a  night  “2v2  intercept”  exercise5,  35  miles  off  the  cost  of  Ensenada,  Mexico. 
Both  pilots  survived  the  incident,  which  is  rare  for  a  spatial  disorientation  (SD)  mishap;  however, 
the  aircraft  was  damaged  beyond  repair,  costing  the  U.S.  Government  approximately  70  million 
dollars.  Flight  incident  recording  data  indicated  a  series  of  extreme  maneuvers  preceding  the 
mishap,  with  high  G-loading.  An  average  of  5.5  Gz  was  sustained  during  the  final  15  s  of  the 


5  2  versus  2  intercept  exercise:  The  pilots  involved  in  this  mishap  were  practicing  maneuvers  wherein  two  aircraft 
investigated  two  other  unidentified  aircraft  that  were  in  their  airspace  and  essentially  escorted  the  unidentified 
aircraft  out  of  the  territory  or  take  necessary  defensive  action. 
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flight.  This  value  peaked  at  7.46  Gz  immediately  prior  to  pilot  ejection,  but  G-induced  loss  of 
consciousness  was  not  indicated.  According  to  MAJ  Thomas  Mondeaux,  an  Aviation  Safety 
Officer  who  evaluated  the  mishap,  “There  was  no  evidence  of  loss  of  consciousness,  because 
control  inputs  were  made  throughout  the  end  of  the  flight,  the  pilot  communicated  and  ejected 
successfully...”  (personal  communication,  September  19,  2011;  Newman,  Lawson,  Rupert,  & 
McGrath,  2012).  Given  that  the  incident  occurred  at  night,  over  the  ocean,  and  with  such  extreme 
forces  acting  on  the  body,  it  is  likely  SD  played  a  role  in  causing  or  exacerbating  this  accident.  A 
re-creation  of  the  aircraft’s  orientation,  instrumentation,  and  pilot  control  input  was  created  by 
NAVAIR  (Naval  Air  Systems  Command)  ASIST  (Aircraft  Ship  Integrated  Secure  and  Traverse 
System)  from  the  DFIR  (Digital  Forensics  and  Incident  Response)  recording.  We  used  the  same 
data  as  NAVAIR  to  simulate  the  final  42.5  s  of  the  mishap  with  the  OMS. 

Data  preparation. 

Data  from  the  DFIR  recording  was  up-sampled  to  100  Hz  using  cubic  spline  interpolation 
(Figure  48).  This  data  was  converted  to  the  proper  coordinate  frame  and  used  to  generate  a  series 
of  OMS  input  files.  All  files  consider  the  final  42.5  seconds  of  flight  (the  same  time  frame 
considered  in  the  NAVAIR  video  recreation). 

Three  different  sensory  cueing  scenarios  were  produced  in  order  to  dissociate  the 
influence  of  each  sensory  system  on  pilot  orientation  perception.  For  all  three  examples,  we 
assumed  that  the  pilot  was  facing  forward  and  that  the  pilot  lacked  continuous  information 
concerning  his  visual  position  (i.e.,  out-the -window  estimate  of  altitude),  linear  velocity  (i.e., 
moving  visual  scene  over  the  ground)  and  orientation  (i.e.,  a  natural  or  artificial  attitude  clue  or 
horizon  line).  These  assumptions  appear  to  agree  with  the  visual  conditions  and  other 
circumstances  of  the  mishap. 

In  our  initial  discussion  of  the  accelerations  associated  with  the  mishap,  we  consider  only 
the  angular  cues  acting  on  the  aircraft.  G-force  values  are  therefore  ignored  and  are  assumed  to 
be  equal  to  zero  for  the  duration  of  the  simulation.  For  Case  2  (section  “FI  8  mishap  analysis 
[angular  velocity  cues  +  acceleration  cues]”)  we  consider  both  the  angular  cues  and  the 
acceleration  cues  acting  on  the  body.  For  the  final  case,  Case  3  (section  “FI 8  mishap  analysis 
[G-excess  parameter  adjustment]”)  we  use  the  modified  G-excess  parameter  set  (introduced  in 
section  “G-Excess  illusion”)  and  repeat  the  simulation  with  both  angular  and  linear  acceleration 
cues. 


This  section  is  intentionally  blank. 
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Figure  48.  Spline  fits  for  FI 8  mishap  analysis.  (A):  Raw  5  Hz  Gz  data  and  100  Hz  spline  fit. 

Raw  2  Hz  angular  velocity  data  and  100  Hz  spline  fits  for  roll  (B),  pitch  (C)  and  yaw  (D)  angular 
velocity. 
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F18  mishap  analysis  (Isolated  angular  velocity  cues). 


VEST  ■  VISUAL 


Figure  49.  Orientation  perception  during  FI 8  mishap  (angular  velocity).  Isolated  angular 
velocity  cues  (aircraft  acceleration  artificially  set  to  zero,  see  text  for  explanation).  (A): 
Estimated  and  actual  roll  angle  (°).  (B):  Estimated  and  actual  pitch  angle  (°).  (C):  Estimated  and 
actual  yaw  angle  (°). 

Figure  49  shows  model  results  for  the  FI  8  mishap  pilot’s  estimated  perception  of  aircraft 
orientation.  For  this  example,  we  assume  only  angular  velocity  cues  and  gravity  are  acting  on  the 
body.  By  isolating  the  angular  cues,  we  can  determine  if  angular  roll,  pitch,  and  yaw  velocity 
perceptions  alone  (due  to  canal  washout  or  interaction  with  the  internal  estimates  of  gravity  and 
acceleration)  would  produce  appreciable  disorientation  or  illusory  self-motion. 

Results  suggest  that  pilot  perception  of  roll  (Figure  49  A)  and  pitch  (Figure  49B)  angle 
would  be  nearly  accurate  given  these  conditions.  While  the  estimated  yaw  angle  (Figure  49C)  is 
slightly  over-estimated  towards  the  end  of  the  simulation,  the  misperception  is  minor,  and 
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unlikely  to  produce  significant  disorientation  or  provoke  the  pilot  response  witnessed  in  the 
accident  re-creation.  Based  on  the  pilot’s  actual  behavior,  it  is  highly  unlikely  that  he  had  such  an 
accurate  perception  of  the  aircraft’s  roll  and  pitch  angle.  If  the  pilot  realized  that  he  had  become 
inverted  (represented  by  the  dotted  line  in  Figure  49 A),  he  would  have  taken  corrective  action  to 
right  the  aircraft.  Likewise,  if  the  pilot  realized  he  was  nose-down  with  greater  than  60°  of  pitch, 
he  would  have  most  likely  taken  a  different  course  of  corrective  action.  Based  on  these  modeling 
results,  we  can  therefore  conclude  that  the  angular  rates  of  the  mishap  alone  cannot  account  for 
significant  disorientation  of  the  pilot.  Below,  we  expand  our  consideration  beyond  linear 
acceleration. 

F18  mishap  analysis  (Angular  velocity  +  linear  acceleration  cues). 
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Figure  50.  Orientation  perception  during  FI 8  mishap  (angular  velocity  and  linear  acceleration). 
(A):  Estimated  and  actual  roll  angle  (°).  (B):  Estimated  and  actual  pitch  angle  (°).  (C):  Estimated 
and  actual  yaw  angle  (°). 

Once  we  consider  the  G- forces  acting  on  the  body  during  the  mishap,  the  OMS  predicts  a 
significant  misperception  of  spatial  orientation  (Figure  50).  A  recreation  of  the  pilot’s  actual  and 
estimated  orientation  is  reproduced  below  to  help  visualize  the  extreme  disorientation  predicted 
by  the  OMS  (Figures  51,  52,  53).  This  recreation  displays  the  mishap  in  2-s  increments  and 
corresponds  with  the  plots  in  Figure  50. 
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Figure  52.  Actual  and  estimated  pilot  orientation  during  F18  mishap  (Part  II  of  III). 
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Figure  53.  Actual  and  estimated  pilot  orientation  during  F18  mishap  (Part  III  of  III). 

The  model  predicts  that  the  most  extreme  errors  in  attitude  perception  would  have 
occurred  about  the  roll  axis  (Figure  50).  The  OMS  predicts  that  from  about  16  s  into  the 
simulation  until  28  s  have  elapsed,  the  pilot  would  feel  upright,  even  though  he  was  actually 
inverted  or  nearly  inverted  (roll  angle  >  90°).  Errors  in  roll,  pitch,  and  yaw  perception  are  plotted 
in  Figure  54  along  with  the  G  level  at  each  time  step.  Predicted  roll  error  peaks  at  24  s  (143° 
error)  and  immediately  prior  to  ejection  (225°  error)  (Figure  54B).  From  29  to  31  s,  as  the  pilot 
“unloads”  some  G  (6  Gz  4  Gz),  accurate  roll  and  pitch  perception  begin  to  recover  somewhat 
(Figures  54A,  54B,  and  54C).  Roll  perception  errors  drop  by  almost  50%,  and  pitch  errors  drop 
by  almost  90%.  At  32  s,  in  an  attempt  to  either  gain  attitude  or  stabilize  the  airplane,  the  pilot 
begins  to  pull  back  on  the  stick  once  again.  The  aircraft  reaches  and  sustains  more  than  6  Gz  for 
the  remainder  of  the  simulation.  Roll  and  pitch  perception  degrade  again,  and  roll  misestimation 
reaches  its  maximum  value  (225°  error). 


63 


The  OMS  confirmed  that  pitch  perception  was  highly  compromised  during  the  moments 
preceding  the  mishap  (Figure  50B).  Maximum  misestimation  of  pitch  angle  coincided  with  the 
16  ->  30  s  time  interval  where  roll  perception  was  most  erroneous.  During  this  time,  true  aircraft 
pitch  angle  approached  60°  nose  down,  with  an  average  pitch  angle  greater  than  30°.  The  model 
predicted  that  pitch  perception  would  indicate  an  orientation  that  is  approximately  upright, 
resulting  in  profound  disorientation.  Pitch  error  peaked  at  28  s  (39°  pitch  error). 


Figure  54.  Errors  in  roll,  pitch,  and  yaw  perception.  Values  in  RED  are  negative.  (A):  G  loading 
(Gz).  (B):  Roll  angle  error  (°).  (C):  Pitch  angle  error  (°).  (D):  Yaw  angle  error  (°). 
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Errors  in  heading,  or  yaw  angle,  do  not  at  first  appear  to  cause  significant  disorientation. 
However,  if  we  step  through  the  animated  frames  presented  in  Figures  51,  52,  and  53,  we  notice 
that  misestimation  of  yaw  angle  is  more  insidious  than  it  first  appears.  When  combined  with  roll 
and  pitch  misestimation,  yaw  errors  serve  to  exacerbate  perceptual  errors.  This  is  especially 
dangerous  when  the  pilot  is  partially  or  fully  inverted.  As  perception  of  yaw  angle  began  to 
degrade  from  19  to  24  s,  the  pilot’s  incorrect  perception  of  being  upright  was  reinforced,  even 
though  he  was  actually  rolled  greater  than  100°. 

Based  on  these  model  predictions,  we  conclude  that  a  significant  spatial  disorientation 
event  occurred  from  16  to  28  s  into  the  simulation.  This  corresponds  to  the  reports  of  the 
aviators,  who  said  they  experienced  SD  prior  to  ejection.  The  event  was  likely  initiated  by  the 
large  roll  misestimation  that  occurred  at  time  t  =  14  s  as  the  pilot  began  to  pull  significant  Gs. 
The  added  G  forces  reinforced  the  feelings  of  being  upright,  as  the  aircraft  began  to  invert.  A 
complete  loss  of  spatial  orientation  followed  this  inversion.  At  t  =  30  s,  perception  began  to 
improve  due  to  G  unloading,  however,  it  was  likely  too  late  to  recover  the  aircraft. 

F18  mishap  analysis  (G-excess  parameter  adjustment). 
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Figure  55.  Comparison  of  orientation  perception  during  FI  8  mishap  with  and  without  G-excess 
parameter  adjustment.  (A):  Estimated,  G-excess  and  actual  roll  angle  (°).  (B):  Estimated,  G- 
excess  and  actual  pitch  angle  (°).  (C):  Estimated,  G-excess  and  actual  yaw  angle  (°). 
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For  completeness,  we  included  a  simulation  that  uses  the  modified  G-excess  parameters 
(Figure  55).  Since  we  do  not  have  information  regarding  pilot  head  movements  and  head 
orientation,  we  cannot  fully  utilize  the  G-excess  parameter  set.  Nevertheless,  limited  head 
movements  would  be  expected  during  the  high-G  forces  preceding  the  mishap,  and  results  from 
the  G-excess  simulation  are  very  much  in  agreement  with  the  results  presented  in  the  previous 
section  (“FI 8  mishap  analysis  [angular  velocity  cues  +  acceleration  cues]”).  The  G-excess 
parameter  set  predicts  a  slightly  reduced  perception  of  pitch  angle  (Figure  55B).  In  this  particular 
case,  the  G-excess  results  do  not  change  the  estimated  cause  or  severity  of  the  overall  spatial 
disorientation  event. 

Visualization  tool. 

A  visualization  tool  was  created  as  a  companion  to  the  NAVAIR  accident  recreation 
video  (Figure  56B).  This  tool  allows  the  user  to  visualize  the  angular  rates,  G  level,  and  actual 
and  predicted  pilot  orientation,  moment-by-moment,  throughout  the  simulation.  Screen  shots  of 
the  companion  tool  were  used  to  generate  Figures  51,  52,  and  53. 


Figure  56.  F18  mishap  visualization  tools.  (A):  NAVAIR  re-creation.  (B):  Companion 
perception  modeling  visualization  tool. 

Further  considerations. 

Simulation  results  could  be  improved  in  the  future  by  incorporating  real-time  eye  and 
head  tracking  recordings  of  pilot  motion  during  the  mishap.  These  variables  would  allow  the 
model  to  better  estimate  the  importance  of  G-excess  and  cross-coupled  effects  and  better 
determine  visual  cues  derived  from  the  aircraft  instrument  panel.  For  example,  if  we  knew  that 
the  pilot  looked  at  the  attitude  indicator  at  time  t  =  34  seconds,  we  could  model  both  the  head 
movement  required  to  accomplish  that  action,  and  input  a  visual  cue  to  the  model  that  represents 
an  improved  awareness  of  overall  orientation. 
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Conclusions 


A  spatial  OMS  and  perception  analysis  toolbox  (Perception  Toolbox)  were  developed 
and  refined  to  aid  in  the  processing,  simulation,  and  visualization  of  human  perception  in 
response  to  three-dimensional,  complex,  multisensory  motion  stimuli.  The  OMS  was  capable  of 
reproducing  human  perceptual  response  to  more  than  a  dozen  classic  laboratory  sensory 
paradigms,  and  it  successfully  predicted  several  common  spatial  disorientation  illusions  (viz., 
Coriolis  cross-coupling,  post-turn,  somatogravic,  and  G-excess). 

The  Perception  Toolbox  successfully  integrated  novel  visualization  techniques  with 
specialized  tools  for  modeling  high-G  perceptual  environments.  The  OMS  and  six  other  classic 
perception  models  were  programmed  into  the  Perception  Toolbox  to  facilitate  comparison  with 
previous  research  and  modeling  results. 

The  OMS  and  Perception  Toolbox  were  used  to  perform  a  case  study  of  an  F18  mishap. 
Model  results  imply  that  profound  SD  was  present  throughout  the  final  42.5  s  of  flight.  The  case 
study  highlights  the  strength,  functionality,  and  crucial  integration  of  the  OMS  and  the 
Perception  Toolbox.  Together,  these  modeling  tools  elucidated  the  time  course  of  SD  during  the 
mishap,  the  sensory  cues  that  played  a  pivotal  role  in  producing  the  SD,  and  the  likely  role  of  G- 
excess  effects.  The  modeling  tools  also  helped  to  visually  represent  the  disorientation  as  it 
unfolded  in  real  time. 

The  Perception  Toolbox  and  the  OMS  were  also  designed  to  accommodate  additional 
sensory  information  that  is  not  presently  available  from  the  cockpit  recorder.  In  the  future,  head, 
eye,  and  gaze  tracking  data  could  be  directly  integrated  into  the  model  without  model 
modification  and  used  to  further  improve  simulation  accuracy,  once  they  become  available. 

The  model  potentially  has  wide  applications  for  aviation  mishap  modeling  and 
simulation.  For  example,  the  human  inability  to  accurately  detect  and  estimate  certain  kinds  of 
vertical  motion  is  relevant  for  vertical  take-off-and  landing.  Since  the  model  concerns  human 
orientation  perception  rather  than  flight  control  per  se,  the  model  also  has  the  potential  to 
positively  impact  other  aspects  of  orientation  perception,  e.g.,  by  modeling  of  human  perception 
of  standing  balance  and  equilibrium  in  the  terrestrial  setting  or  following  the  sort  of  G  transitions 
astronauts  face. 

The  applications  of  the  OMS  described  in  this  paper  are  being  expanded  upon  and 
elucidated  by  our  other  work.  For  example,  Newman,  Lawson,  McGrath,  &  Rupert  (2014) 
elaborated  upon  the  need  for  the  addition  of  gaze  prediction  capabilities  to  the  model,  which  can 
be  achieved  through  the  use  of  eye  tracking  technology.  The  incorporation  of  gaze  tracking  into 
the  model  would  make  it  possible  to  improve  inferences  concerning  the  visual  focus  of  attention 
of  the  pilot  during  the  onset  of  SD  (Newman,  Lawson,  Rupert,  &  McGrath,  2014).  McGrath  et 
al.  (2015)  discussed  advanced  SD  training  applications  that  would  be  feasible  through  the  use  of 
a  simulator  that  takes  into  account  the  pilot’s  perceived  attitude  and  location  of  the  ground,  to 
teach  pilots  SD  detection  and  recovery  techniques.  In  addition  to  the  reports  above,  Lawson  et  al. 
(2015)  and  McGrath  et  al.  (2015)  describe  the  model’s  ability  to  analyze  post-hoc  flight  mishaps 
to  determine  causation,  which  may  contribute  to  the  development  of  a  potential  future  in-flight 
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spatial  disorientation  detection  capability.  Finally,  the  description  of  the  OMS  in  this  report  is 
supplemented  by  the  description  of  the  interface  and  instructional  user-guide  presented  in 
Newman,  Lawson,  Rupert,  Thompson,  and  McGrath  (2014). 

Increased  understanding  of  the  situations  that  induce  spatial  disorientation  and  contribute 
to  mishaps  should  tremendously  decrease  the  number  of  spatial  disorientation  mishaps  in  the 
future.  The  model  described  in  this  paper  was  initially  developed  by  Newman  (2009)  as  a 
Master’s  Thesis  at  the  Massachusetts  Institute  of  Technology,  then  revitalized  and  expanded 
under  a  USAARL  In-House  Independent  Laboratory  Research  Effort  to  the  state  that  it  is 
reported  here  and  elsewhere  (Newman,  Lawson,  Rupert,  Thompson,  &  McGrath,  2014).  The 
model  is  currently  undergoing  many  further  improvements  in  preparation  for  transition  via 
triservice  (Defense  Health  Program)  research  and  Small  Business  Innovation  Research  (SBIR). 
Subsequent  reports  will  detail  further  model  improvements. 

Recommendations 


While  our  model  has  expanded  the  perceptual  phenomenon  that  can  be  predicted,  a  number  of 

future  model  refinements  are  recommended,  some  of  which  are  underway: 

•  Improved  head  movement  model:  The  current  head  movement  model  included  in  the  OMS 
simplifies  the  coordinate  transformation  between  the  reference  frame  of  the  input  stimuli  and 
the  head.  A  more  complex  model  based  on  true  head-neck  kinematics  should  be  developed  to 
improve  simulation  results  and  accuracy. 

•  Integration  of  canal  and  otolith  thresholds:  The  OMS  needs  to  model  the  mechanical 
thresholds  of  the  semicircular  canal  or  otolith  organs.  Integration  of  thresholds  would 
improve  simulation  of  SD  illusions  such  as  the  leans. 

•  Improved  visual  sensor  models:  The  OMS  does  not  dissociate  between  focal  and  ambient 
visual  function.  An  improved,  physiologically-validated  vision  model  would  improve  vection 
simulation  results  and  provide  more  realistic  processing  of  visual  sensory  information.  For 
example,  the  model  should  simulate  the  fact  that  vection  onset  is  not  always  immediate,  and 
that  a  subject  cannot  always  estimate  a  rotating  or  translating  visual  velocity  cue  accurately. 
Visual  model  refinements  will  improve  the  performance  of  the  model. 

Additional  research  is  recommended  to  improve  the  model: 

•  G-excess:  Further  experimentation  would  be  beneficial  to  validate  the  G-excess  parameter 
modification  discussed  in  the  section  “G-Excess  illusion.”  For  example,  further  data  is 
desired  concerning  the  time  course  and  dynamics  of  the  illusion  under  different  G 
environments.  Such  work  is  already  underway  as  a  result  of  the  SBIR  effort  mentioned  above 
(see  Conclusions  section).  Some  of  these  results  will  be  published  soon. 

•  Coriolis  cross-coupling:  Further  data  on  the  time  course,  magnitude,  direction  and  duration  of 
the  Coriolis  cross-coupling  illusion  would  help  to  validate  several  of  the  predictions 
presented  in  this  report. 

•  Vertical  Motion  Perception:  In  order  to  validate  the  limbic  coordinate  frame  velocity  and 
displacement  integrator  aspects  of  the  model,  a  large  amplitude  vertical  vs.  horizontal  motion 
experiment  is  recommended.  The  experiment  should  test  vertical  motion  with  the  human 
body  z-axis  aligned  with  and  perpendicular  to  the  gravity  axis,  measuring  perceived 
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displacement  amplitude  and  phase.  An  identical  experiment  could  be  performed  for 
horizontal  motion  with  the  body  z-axis  aligned  with  and  perpendicular  to  the  direction  of 
displacement. 


This  section  is  intentionally  blank. 
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Appendix  A.  Spatial  Rotations 


A.l  Coordinate  Systems 


+Y 


w 


Figure  A1  Head  and  World  coordinate  frames. 


We  define  a  right  handed  coordinate  system  relative  to  the  world  (Xw,  Yw,  Zw)  and  the 
head  (X,  Y,  Z)  (Figure  Al).  It  is  assumed  that  the  semicircular  canals  and  otoliths  are  situated  at 
the  center  of  the  head  and  align  with  the  naso-occipital  X,  interaural  Y,  and  dorsoventral  Z,  axes. 
For  computational  simplicity,  angular  velocity  and  linear  acceleration  inputs  to  the  vestibular 
model  are  processed  in  the  egocentric  head-fixed  frame.  It  is  often  necessary  to  transform 
quantities  between  reference  frames.  Gravity,  for  instance,  is  inherently  defined  in  world 
coordinates  yet  needed  for  the  GIF  (gravito-inertial  force)  calculation  performed  in  the  head 
axes.  As  we  rotate  and  translate  about  in  space,  a  novel  description  of  the  relationship  between 
these  coordinate  frames  is  therefore  required. 

A.2  Quaternion  Representation 

The  quaternion  provides  us  with  a  useful  notation  for  representing  spatial  rotations. 
Quaternions  eliminate  gimbal  lock,  reduce  numerical  storage  from  nine  to  four  digits  (nine  being 
the  typical  representation  of  a  rotation  matrix),  and  increase  computational  stability.  A 
quaternion  representation  for  our  model’s  coordinate  frames  and  vector  rotations  is  therefore 
preferred.  We  can  define  a  unit  quaternion  in  the  following  form: 
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q  =  q0+qli  +  q2j  +  q3k 

with  i2=f=k2=- 1 

In  order  to  update  the  quaternion  vector  as  we  rotate  in  inertial  space  the  initial 
quaternion  q0  1  must  be  integrated  with  respect  to  the  angular  velocity  input, 

(bit )  =  \oJx(t),OJY(t),OJz  (0]r  ■  A  stable  algorithm  to  perform  this  integration  was  developed 
by  Fang  &  Zimmerman  (1969). 


q0  = — ^  (qi^x 

q\  ~ 

=  ^iqio^x 

q?> =  ~  (q  i  ■ 


+  q^cOy  +  q^(oz )  +  k^q^ 


-q0((>Y  -q2coz)+ k^q\ 


+  q3a>Y  qxcoz 


)  +  k  Aq  2 


■  (')y  +  €1,(0,  )  +  k XCj , 


(1) 

(2) 

(3) 

(4) 


A -\-(q02 +q2 +q22 +q2)  (5) 

Integrating  Equations  1-5  yields  a  complete  time  history  for  the  quaternion  vector  q  . 
This  particular  formulization  uses  an  algebraic  constraint  to  minimize  the  constraint  error.  For 
alternate  integration  schemes  using  normalization  or  derivative  constraints,  refer  to  Fang  & 
Zimmerman  (1969).  Constraint  errors  represent  a  non-orthonormality  in  the  transformation 
matrix  and  are  thus  extremely  problematic  for  the  decomposition  of  vectors.  The  proportionality 
constant  ensures  stability  such  that  k  >  0  and  the  product  hk  <  1 ,  where  h  is  defined  as  the 
integration  time  step.  A  value  of  k  =  0.8  -  0.9  worked  best  for  our  input  file  sample  rates  and 
Simulink  ODE45  differential  equation  solver. 

The  integrated  quaternion  now  provides  us  with  all  the  necessary  information  to 
transform  vectors  between  the  head  and  world  coordinate  frames.  At  each  time  step  a  rotation  of 

the  gravity  vector  gw  =  \gXw’SYw’8zwf  is  accomplished  with  the  transformation  matrix  T; 


1  The  initial  quaternion  is  calculated  based  on  the  initial  gravitational  state.  Assuming  the  subject  is  oriented  upright, 
inline  with  the  gravitational  vertical,  and  in  a  1G  environment  —  [o,o, -if  and^Q  —  [i,o,o,of . 
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2  2  2  2 


2(qlq2+q0qi)  2  (qxq3-q0q2) 


q0  +qx  -q2  -q3 
T=  2(qxq2-q0q3) 

2(q0q2+qlq3) 


q02  -  q2  +  q2  -  q3  2  (q0qx  +  q2q3 ) 
2(q2q3  -  q0qx )  q2  -  q2  -  q2  +  q2 


such  that  the  current  direction  of  gravity  in  head  coordinates  can  be  expressed  as  the 
premultiplication  of  gw  with  T. 


g(t)  =  T(q)-gw 


(6) 


Likewise,  we  can  integrate  Equations  1-5  with  respect  to  the  internal  estimate  of 
angular  velocity,  (b ,  and  obtain  a  similar  representation  for  the  estimated  gravity  state. 


(7) 


The  estimated  initial  gravity  vector,  gw  ,  and  the  true  initial  gravity  vector,  ,  are 

assumed  to  be  equivalent.  A  discrepancy  between  these  vectors  would  result  in  a  perceived 
sustained  acceleration  inconsistent  with  actual  human  perception  literature.  In  any  gravity 
environment,  the  CNS  is  therefore  modeled  to  maintain  an  initially  veridical  estimate  of  the 
direction  and  magnitude  of  gravity. 

A.3  Limbic  Coordinate  Frame  Calculation  and  Quaternion  Transformation 


Figure  A2  Limbic  coordinate  frame. 
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The  quaternion  vector,  q ,  from  the  estimated  gravitational  state  completely  defines  the 
limbic  coordinate  frame.  As  shown  in  Figure  A.2,  the  limbic  frame  confines  to  the  same  right 
handed  orientation  and  sign  conventions  of  the  other  coordinate  systems.  The  XL-YL  horizontal 
plane  is  perpendicular  to  the  direction  of  estimated  gravity  and  acts  as  our  natural  plane  of  2D 
navigation.  Just  as  T  was  used  to  transform  vectors  from  world  to  head  coordinates,  its  inverse 

/V 

can  be  used  to  perform  the  opposite  duty.  Driven  by  the  estimated  quaternion,  q  ,  T1  can 
transform  any  vector  from  head  to  limbic  coordinates.  Rearranging  Equation  7,  and  substituting 
the  estimated  acceleration  vector  in  place  of  gravity,  we  obtain; 

T(q)-aL(t)  =  a(t)  <8> 

Where  a,(t)  corresponds  to  the  estimated  acceleration  vector  expressed  in  the  limbic  frame.  Pre¬ 
multiplying  both  sides  of  Equation  8  by  the  inverse  transformation  we  can  solve  explicitly  for 
aL(t). 

T~'  (|)  •  A! )  •  3l  (?)  =  T~'  (f )  •  5(0 

aL(t)  =  T~'(q)-a(t) 

A.4  Visual  position  and  velocity  transformations 

A  direct  conversion  between  the  world  and  limbic  frame  is  not  possible  with  the  current 
quaternion  setup.  To  circumvent  this  potential  problem,  inputs  are  converted  from  world  to  head 
coordinates  and  then  subsequently  from  the  head  to  the  limbic  system.  This  rotation  is  performed 
with  two  transformation  matrices.  The  expressions  for  visual  position  and  velocity  are  shown 
below. 


T-\$)-{nq)-icy) 

A.5  Visual  gravity  and  angular  velocity  transformations 

Visual  gravity  and  angular  velocity  are  transformed  to  head  coordinates  prior  to 
processing.  Rearranging  equation  6  and  substituting  the  visual  inputs  we  obtain  expressions  for 
their  coordinate  frame  transformations. 

T (q) ■  av 

T(q)-gv 
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